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ABSTRACT 


This study tested a model which was developed as a guide for the 
monitoring of an educational system. The specific purpose of the 
Study was to identify variables which should be included in a program 
monitoring approach and to demonstrate how such variables could be 
used within an evaluation system designed to monitor the quality of 
education in on-going school programs. 

theoretical Constructs and perspectives were adopted from 
literature in the areas of program evaluation, social indicator 
development, educational planning, organizational effectiveness, and 
information systems. The model developed contained three major phases 
including: (a) data collection and preparation, (b) inductive inquiry, 
and (c) information. Existing data were utilized, hence only the last 
two phases of the model were applied in the present study. These data 
included information from Alberta Department of Education files and 
from a followup questionaire for an approximate 10% random sample of 
all grade eleven students in the province who had been enrolled in the 
1970-71 school year. 

A total of 20 general indicators of quality education and seven 
SuDJeECt aréa specific indicators were identified from the available 
data. The greatest amount of variance in these indicators of quality 
education was accounted for by seven of the 25 student-based and 
school-based predictor variables utilized. The seven included student 
program, pupil-teacher ratio, sex of respondent, T-SCAT ability score, 


grade nine average, self ability rating, and career plans. Up to 56% 
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of the variance was accounted for by the first five of these predictor 
variables; this percentage in the case of predicting tne high school 
average mark. Overall, the high school program followed by the 
Student accounted for the most variance in the quality indicators. 
Although predictability was lower for the externally measured 
indicators, up to 30% of the variance was accounted for, even though 
these followup data had been gathered five years after the students 
left school. 

Provincial norms were computed for each indicator of quality 
education. The general quality indicators provided information such 
as: that 65% of the student group achieved a high school diploma and 
that 30.5% attained matriculation status. The general indicators of 
quality education were disaggregated on the basis of sex of student, 
previous achievement, career plans, and Student program. Further 
Gisaggregations of the indicators based on the selected student 
Ehoabeeech rst es were carn .ed CULL Within Gach of the four, student 
programs. Additional analyses were done using two of the general 
indicators (percentage of students achieving a high school diploma and 
the average high school mark) to enable comparisons among nine school 
districts and 12 schools which had been selected for illustrative 
purposes. 

A general conclusion of the study was that macro evaluations of 
school programs, of the type demonstrated, have potential in providing 
useful information for educational program planners and decision 


makers at the system level. 
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CHAPTER 1 


INTRODUCTION 


Evaluation of school programs at the general systems level 
requires consideration of approaches advanced in educational planning, 
needs assessment, information systems, as well as curriculum 
evaluation. Concepts developed within the study of organizational 

BeeCiemvenecS sAnGmeneCmsOCrla MEIC CAcOmm 1 eeOGaAcU nee musta lSOmbpe 
considered. 

Evaluation activities which are perceived to be summative at one 
level of inquiry may become formative when the scope of inquiry 
meManges eoTineludesaviargersiramelorurererence,orsaimore general 
system. The study reported herein focuses on an approach which is 
formative evaluation at the general level of "Schooling". The 
approach proposed iS a continual, PesTOGLemnCeebackwOmmtTORMate on, 
providing formative evaluation of an educational system, its on-going 
instructional programs and its major sub-systems. Such an approach 
Can provide information regarding the quality of the processes and 
outcomes of schooling and identify areas requiring more in-depth 
evaluation, such aS a curriculum evaluation of a specific subject area 
or an organizational effectiveness study of a school or school system. 

The level of evaluation reported in this study is 
macro-evaluation, which is the general monitoring OG aldaeaing .On san 
educational system, in the sense of a periodic check to determine the 


effectiveness of that system and the quality of its outcomes. 
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The purpose of this study was to test a model for monitoring an 
educational system which could provide periodic information (ideally 
on an annual basis) at the provincial, district, and school levels on 
whaen to base general Curriculum and resource allocation decisions. 
Such a system could produce selected information taken froma 
student-based sampling of system files, public opinion, student 
opinion, student self-report information, and educator opinion. 

Ideally, such a system would identify the types and sources of 
information and outline the analytical strategies and the framework 
for the presentation of information which could provide each level of 
the provincial educational system with: 

a. indications of the general quality of education, 

b. indications of the quality of education within each subject 

area vis-a-vis provincial objectives, 

©. Guidance for policy and resource allocation decisions, and 


Ceol ECCelOne nore UmeneCrean-ACpDEn CuUBETCULUM evaluations. 


Justification for the Study 

Very little systematically collected information of a 
longitudinal nature iS available to school authorities to guide their 
curriculum and resource allocation decisions at the general or "macro" 
level of the operation of schools and school systems. "One-shot" 
surveys or opinion polls are conducted, advisory groups are 
established, petitions are received from various pressure groups, the 
"pulse" of the electors is felt by politicians, but systematically 


collected evidence of achievement, of quality, of strengths and 
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weaknesses, is almost non-existent. The study reported here attempts 
to examine an evaluative approach which could begin to meet these 


needs. 


Educational Evaluation and Accountability 

A trend over the last two decades has been increasing pressure 
for more accountability in education. There have been the demands for 
closer specification of educational objectives, the implementation of 
program budgeting systems, a redefinition of "schooling" to 
Gistinguish it from "education", and calls for quality indicators of 
educational performance. 

Studies during the 1960's in the United SocbAd eORUHEad ina 
Significant increase in interest in educational evaluation which would 
give indications of worth at the general or “macro” level. The 
Coleman Report of the mid-sixties (Coleman, 1966) raised questions 
about the contribution that various aspects of the school made to 
Seudene achievement. The National Assessment of Educational Progress 
(NAEP) of the United States grew out of recommendations from the 
Exploratory Committee on Assessing the Progress of Education (ECAPE) 
which had indicated that because of the absence of comprehensive data 
... "personal views, distorted reports, and journalistic impressions 
are frequently the main sources of public opinion" (p. 9). Lessinger 
(1970) suggested an "educational accomplishment audit" focusing on 
Skill acquisition and, in reply to concerns related to educational 
costs, wrote: 

It would make much more sense if we moved from the concept of 

per-pupil cost to the concept of learning-unit cost, and 


focused on the cost Of Skzil acquisition rather than on the 
COSEVOR MAalINvaliangchmldsenein Schools: P14 
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The Social Indicator Movement 

Concerns also developed for social systems other than education, 
and in 1971 the Economic Council of Canada called for the development 
of a wide range of social indicators which would give evidence of the 
well Ueing of ali ‘social systems;ini the country; The» Eighth, Annual 
Review of this council (1971) called for the development of a 
monitoring system and the use of feedback mechanisms which would 
provide for continuous reassessment and realization of objectives, 
policies, and programs. The Review makes reference to quantitative 
and qualitative information collected on a time-series basis to 
provide "goal indicators." Such a monitoring system could act as an 
early warning system and provide a bridge between broad abstract goals 
and operational guidelines. 

On an international level, the Organization for Economic 
Cooperation and Development laid plans for "Indicators of Performance 
of Educational Systems" (1973) which were summarized as: 

Goals for educational growth and change in the 1970s should 

be made more explicit and where possible indicators which 

would measure the performance of the educational system, both 

in relation to educational goals as such and the contribution 

of education to the wider social and economic objectives, 

should be established (p. 103). 

More recently, Stake (1976) has observed that "although many 


completed evaluation studies fell short of their promise, the demand 


for evaluation in the United States has not diminished" (p. 9). 


The Alberta Scene 
In Alberta, Worth (1972) espoused a need for effective monitoring 
of school programs to "evaluate our system of schooling ona 


systematic and comprehensive basis" (p. 138). With the implementation 


of Program Planning Budgeting and Evaluation Systems (PPBES) in the 
province, such evaluation was provided for, but later the evaluation 
component was dropped and PPBES became PPBS. Duke (1972), in writing 
about PPBES and educational accountability outlined the need for 
Clearly specified objectives, evaluation of these objectives, and 
communication to the public in terms of "what are we doing?" and "how 
well are we doing it?" 

In response to a need for evaluative information of the sort 
indicated above, steps were taken by Alberta Education in 1970, 
through the Curriculum Evaluation Project (Collett, 1978), to 
determine the relevancy of courses taken by high school students to 
the achievement of their aspirations. The project yielded a data bank 
With potential for analysis beyond the purposes of the original study. 

The data bank for the Curriculum Evaluation Project (CEP) 
contained complete Department of Education records for a group of 
approximately 2700 students randomly selected from the total Alberta 
grade eleven population of 1970-71. In addition, the data bank 
included student feedback of a longitudinal nature (through the 
administration of three questionnaires distributed over a seven year 
period) providing important follow-up data, and individual evaluations 
of schooling. The sample had been structured with sub-samples large 
enough to provide information for individual high schools. The 
resulting data bank is large enough to enable the researcher to 
undertake statistical analyses of variables which could be included in 
an educational monitoring system. 

Continuing concern regarding educational system evaluation is 


Still evident. Recent media reports have focused on the question of 


once again implementing annual departmental examinations for secondary 
students in the province. Such examinations could provide evidence of 
how well individual schools are doing and, more generally, how well 
the schools of the province are doing. The Minister's Advisory 
Committee on Student Achievement (MACOSA) has made recommendations 
regarding the periodic implementation of province-wide achievement 
tests. Such recommendations, however, focus only on student grades 
and overlook other, broadly-based, indicators of achievement and 
evaluations of the effects of schooling. Examples include the extent 
Sfecenievement, Of sysremsgoals and objectives, s student attainment ofja 
hagh school diploma, employability, job Satisfaction, etc. 

In summary, there is an apparent need for an evaluative approach 
which would utilize a cross-section of indicators, rather than one 
which focuses on a single indicator such as grades achieved by 
students. Such an approach would provide a synoptic data base 
(providing a general view of the whole educational system), with a 
Watrtecy, OL indicators On the quality of education. The indicators 
would show strengths and weaknesses in particular subject areas of the 
curriculum as well aS in particular sub-groups of the student 
population. Such information would ideally lead to more informed 


Curriculum and resource allocation decisions. 


The Research Problem 
Within the direction established earlier in the discussion of 
purpose, this research study sought to partially test a model for 
monitoring an educational system and to identify data and data 


handling procedures which could provide an informative and evaluative 
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Set Of general and subject area quality indicators. Such quate 
indicators, when appropriately disaggregated, could be used to monitor 
the provincial educational system and major sub-systems within the 
province, including schools and on-going curricular programs at the 
macro level of evaluation. 

UEREEZangeche Long L_LEeudinateandeprlerdata within che Curriculum 
Evaluation Project data pank, the problem was to identify variables 
which should be included in a program monitoring approach and to 
demonstrate how such’ variables couid bevused within an evaluation 
system designed to monitor the quality of education in on-going school 


programs. 


Sub-Problems 
In addressing the above problem, the research was directed toward 

the following more specific sub-problems: 

ig to identify general and subject area indicators of quality: 
education; 

Bic to identify student—based and school-based variables which 
ACCOUNT, LO STOoneticane Vvarrance in Ehe andicatons Of quality 
education; 

Sie to determine, for selected indicators of quality education, the 
total variance which is accounted for by the student-based and 
school-based variables and the explanatory power of each of these 
variables; 

4, €O Compute provincial nonums for each indicator of quality 
education; 


Be to identify, for selected indicators of quality education, 
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differences among groups of students classified on the basis of 
student programs and student characteristics; 
6% to develop profiles for selected schools and diStricts using the 


Various indicators of quality education. 


Delimitations 

THeUsStnG Senew daca bankeory Che Curr culum Evaluation Project, enis 
research is delimited to thendaes that were included in that study. 
Specifically, the CEP was delimited to the high school programs of 
Alberta. Grade eleven students registered in Alberta schools during 
the school year of 1970-71 were identified as the population to be 
studied. Data sources were delimited to Department of Education 
student records, school A-Card reports, a grade 11 student 
questionnaire, and two followup surveys. 

TNS Meneses wasetunehes delamipeced GO Ehe variables Within CEP 
which could be identified aS potential quality indicators, and to 
others which potentially would be required to disaggregate the quality 
indicators., AS a result, a number of the questionnaire variables 
Whichdecaleawaeh Srugentvoprnren icConcerning —Ehe quality of their 
educational experience were eliminated as were the variables concerned 
with student views about the importance of the various goals and 
school programs. 

On the other hand, student outcome data such as the distribution 
of marks, which were in the data bank but not used by CEP, were 


SCueeEeCauEOG eels On Baim SmS Guel ya. 


Limitations 
social research, especially of the longitudinal type, is limited 
by the almost infinite number of unidentified contaminating variables 
which cannot be taken into account. Nevertheless, if the results of 
Such longitudinal research are interpreted with this in mind, progress 
can be made toward understanding relationships between earlier and 


later events. 


Limitations of the Curriculum Evaluation Project 
Major limitations of the Curriculum Evaluation Project included: 
a. the effectiveness of the questionnaire used in collecting 
accurate and honest feedback from students, and 
Deethe iess than optimal return rate (27%) for the third 


questionnaire mailed to students five years after they left 


Lv hieschoowr, 


Further Limitations 

The limitations imposed on any study which depends upon existing 
data were operative in the present study. For example, no changes 
could be made in the data gathering procedures used. Another 
limitation was imposed by the age of the data which were collected 


three years prior to completion of the present study. 


Assumptions 
The underlying assumption of this research was that because of 
the complexity of the general area of evaluation at the macro level, 
one way to aid advancement and further development of macro evaluation 


was to undertake one, which in turn would provide a base line study 


eee iiieer ai 
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for subsequent refinements in the development of a viable system. 
Furthermore, it was assumed that accurate evaluative judgements 
of goal achievements can be collected from ex-clients of the school by 


means of a questionnaire. 


DeEranztcvons 

The; following definitions of terms are included ‘to assist the 
reader in understanding the report of this study. 

Monzeon. ~The perrodi1e icoimecezon and presentation of information 
on the quality of education generally, and on the quality of 
sub-systems and curricular programs. 

Audit. Used synonymously with monitor, excluding one recent 
usage of the term which focuses on a Specific type of meta-evaluation 
process. 

Quality Indicators. A comprehensive, yet limited set of 
policy-related and goal-based statistics (variables) which can be 
collected periodically and disaggregated to oie wlevell of significant 
sub-systems and client sub-groups, in order to reveal trends and 
SIUMCTUALLONS. 

Disaggregate. To compute the numerical value of a quality 
indicator for a specified sub-group of students. 

Breognant. snes Ccunrmiculalofieredby a School, Won va subyeceraneca 
component of that curricula. In this sense, the "school program" may 
be referred to, or the "mathematics program" (as an example) may be 
referenced. Alternatively, program is used to refer to a group of 
curricular experiences (courses) taken by a student or group of 


students, which comprises their "student program". 


Organization of the Thesis 

The first chapter of this thesis has presented the purpose and 
justification for the study. The research problem was outlined along 
with specific sub-problems which guided the research and provided the 
structure for the remainder of the thesis. The chapter included the 
deliamitations, limitations, assumprions, and definitions associated 
Wath the estuday. In Chapter 2 the theoretical background and 
conceptual framework of this research is presented, along with a model 
for monitoring educational systems, designed by the author. The model 
for monitoring an educational system iS proposed as a type of on-going 
exploratory research, providing formative evaluation at the system 
Level.) sneetheme: ofethe model dsrarcyclicaisamonitoringsior the 
educational system as well as the process of monitoring. 

In Chapter 3 the Curriculum Evaluation Project which provided the 
data bank for the present study, is reviewed. The focus of the review 
is on the sample structure and the data collection methodologies used 
Sum OmmOUmelcmarnCInG CmOrmcla cpanel CUlahmDnOjeCt. 

The methodology of the present study is described in Chapter 4. 
The research questions are Stated and the research procedures for each 
question reviewed. 

Chapters 5 and 6 present the results of the study. Chapter 5 
deals with the findings associated with the first three sub-problems 
which represent the inductive inquiry phase of the study. The 
sub-problems were concerned with: 

a. identifying both general and subject area quality indicators, 


b. identifying the student-based and school-based variables which 
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determining the total variance accounted for, along with the 
relative explanatory power of each of the student-based and 


school-based variables. 


Chapter 6 presents the results for the last three sub-problems 


which form the information phase of the study. These sub-problems 


were concerned with: 


a. 


o~ 


Le 


the computing of provincial norms for both general and subject 
area quality indicators, 

identifying the different levels recorded on selected quality 
indicators by groups of students classified on the basis of 
student programs and student characteristics, and 

developing profiles for selected schools and districts using 


the quality indicators. 


Finally, Chapter 7 summarizes the study and presents the study 


conclusions and implications. 
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CHAPTER 2 


CONCEPTUAL FRAMEWORK AND MODEL DESIGN 


The conceptual framework for the study grew out of a particular 
perspective on social research generally. Within this general 
research orientation, certain perspectives on evaluation, along with 
concepts and strategies from a variety of other disciplines, 
contributed to the development of the model presented in this chapter. 

Since the macro evaluation perspective adopted in the present 
study relies on the consideration of brief summary data, quantitative 
evaluation has been emphasized. Evaluative perspectives and 
technigues having a qualitative emphasis have not been reviewed in 
detail. The assumption made was that these techniques are more 
appropriate for specific and detailed evaluation than they are for 
general "macro-level" monitoring. 

In the firSt part of the chapter, a brief elaboration of the 
general research orientation taken in the study is presented. Second, 
a variety of concepts is reviewed in order to assist in the 
identification and description of the particular evaluation 
perspective adopted. Third, several strategies taken from the 
evaluation literature and from other fields of study concerned with 
evaluation are presented. Examples of these fields of study are 
organizational effectiveness, educational planning, information 
systems, and social indicator development. In the fourth section of 
this chapter, concepts and strategies are brought together ina 


presentation of a systems evaluation perspective within which the 


cS 


model was developed. A model which could be used to guide program 


monitoring at the system level is then presented. 


Evaluative Research: a Philosophical Viewpoint 

Within the domain of general scientific inguiry, Worthen and 
Sanders (1973) demonstrate that considerable overlap exists between 
the two fields of educational research and educational evaluation. 
Basic research seeks to increase knowledge, and applied research seeks 
SOLUcLONS COepEOobrems. | Evaluation, Like applied research; collects 
information but toward a specific question concerned with the value of 
something. While a number of authors (including Worthen and Sanders) 
have identified characteristics which distinguish between research and 
evaluation, the theoretical orientation taken in this study is based 
OVE CVeeSMELARRELT OS DeEWSen BeScarnch andvevaluation LTatner than on the 
differences. 

His ee reported herein is the type of research which has 
variously been referred to as formative research evaluation, 
sociological inquiry, or inductive research. The following paragrapns 


explain. 


Inductive Sociological Research 

A developing view in the field of social research is that one 
legitimate objective of data analysis should be the discovery and 
formulation of propositions (Sonquist, 1970). According to this view, 
social research has an exploratory role to play as well as the more 


traditional role of hypothesis testing. Sonquist states: 


| ee > [aete 


The problem is one of determining which of the variables for 
which data have been collected are actually related to the 
phenomenon in question, and under what conditions and through 
which intervening processes, with appropriate controls for 
Sspuriousnessie(ps) 1)% 26 
Rutman (1977) describes the characteristics of "formative" research as 
an inductive approach aimed at discovery. He suggests that a 
formative research evaluation ''...is considered as an on-going process 
which guides decisions about program changes" (p. 71). 
Writing of "sociological inquiry" as "nonexperimental inquiry 
directed toward the study of relations among 'social variables'" 
(p.9 144), Kerlinger (1979) states: "Sociological-inguiry, then, is a 
large and highly significant part of contemporary behavioral research" 
Core 157) 2 
In the present study, the orientation described above is evident 
in two ways: 
a. aS an exploratory research project aimed at discovering 
appropriate ways of monitoring on-going school programs, and 
b. in the adoption of an inductive inquiry phase within the 
proposed program monitoring model. 
Phe omsCcConun DOincens clLaAvOratced Oh latereainathe Presentationxor the 


model. Liew tocCecTON mamiUunbeChmOhmeOncepeSmuwhrchmcananeLDm tO 


identify and describe various evaluation perspectives are reviewed. 


Identifying and Descriptive Evaluation Concepts 
What is the scope, the focus, the domain of evaluation as a field 
of study? A review of the literature over the last 20 years shows a 


wide variety of answers to this question. Differing perspectives, 


determined partly by a particular theoretical orientation and partly 


by the type of evaluative problem confronted, have contributed not 
only to different descriptions of what evaluation is, but also to 
considerable confusion in terminology. 

A number of well known theorists in the field of evaluation such 
as Taylor and Cowley (1972), Worthen and Sanders (1973) saHouse. (#973) , 
and Payne (1974) have gathered writings which present a variety of 
perspectives in order to clarify viewpoints and terminology through 
comparison. Other authors have developed various typologies in order 
to rationalize evaluation as a field of study. Some classifications 
by authors such as Stufflebeam (1968) or Anderson and Ball (1978) 
suggest that evaluation activities could be categorized based on the 
type of decision served. Morris and Fitz-Gibbon (1978) grouped 
evaluation perspectives on the basis of emphasis placed on goals, 
decisions, or research, etc. Hastings (1969), Anderson and Ball 
(1978), and Rossi, Freeman, and Wright (1979) have developed 
two-dimensional matrices for use in categorizing evaluation 
perspectives. 

The different concepts associated with the field of evaluation 
are too many and too complex to be handled in a Single matrix. The 
following section reviews and discusses a number of underlying, and 
sometimes overlapping, evaluation concepts that have been identified 
by various authors. Many of these concepts deal either with different 
purposes or different methodological emphases associated with a 
particular evaluation. In some cases purpose and methodology are 
interrelated. 

These concepts were used to help identify the particular 


perspective of evaluation used within this study. 


Goal and Role 

A major contribution to understanding the purpose of evaluation 
has been made by Scriven (1967) in his discussions of the role and 
goalsor evaluation: “The goallor evaluation is to determine: the worth 
of an educational unit. As interpreted by MacKay and Maguire (1971), 
the role of evaluation is broken into two broad categories by two 
other concepts advanced by Scriven, namely formative and summative 
evaluation. The major purpose of formative evaluation is improvement. 
It is an evaluative approach suited for the development of new 
techniques or programs, or as an on-going developmental approach. 
Summative evaluation, on the other hand, has been defined as a "final" 
or terminal evaluation of the worth or effectiveness of competing 
instructional programs. Glass (1972) seems to equate formative 
evaluation with the role of evaluation, as do MacKay and Maguire, but 
treats summative evaluation as a further elaboration of the goal of 


evaluation. 


Goal/Objective Orientation 

Different perspectives on evaluation have often been based on 
different views as to the role that goals and/or objectives should 
play in the evaluation. Positions range from a pure application of 
Provus' (1971) discrepancy model of evaluation to the goal-free 
position advanced by Scriven (1972). 

Intrinsic and payoff evaluation. A pair of concepts advanced by 
Scriven (1967) and discussed for example by Tyler (1967) and Friesen 
(1971) differentiate between intrinsic and payoff evaluation. An 


emphasis on the effects of an instructional unit relates to a "payoff" 
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evaluation. Intrinsic evaluation however focuses on content, goals and 
rocedures of the particular educational unit under study. 

Discrepancy evaluation. Primarily under the influence of Tyler 
(1969) and Provus (1971), evaluation was to focus on developing 
objectives, identifying the extent of achievement of those objectives 
and determining the degree of shortfall. Guba (1969) has criticized 
this kind of an approach as focusing on behavioral changes and as a 
result placing an emphasis on terminal evaluation. Calling into 
question the practicality of focusing on goals, March (1972) observed: 
" ..it seems to me perfectly obvious that a description that assumes 
goals come first and action comes later is frequently radically wrong" 
(p. 420). He goes on to suggest alternatively, that evaluation needs 
to examine what was done in terms of what is currently believed to be 
important. 

In spite of criticism, emphasis on goals or objectives has 
continued to varying degrees and is involved, to some degree, in most 
perspectives on evaluation. Terminology varies, with Stake (1967) 
introducing the concepts of intended and observed outcomes, and others 
using the term "impact" which is used as a longer range goal than an 


outcome. 


Decision Types Served 

The focus of the CIPP evaluation model developed by Stufflebeam 
(1971) is the type of decision to be served by the evaluation 
endeavor. Stufflebeam's model develops in detail: context, input, 
process, and product evaluation for the purpose of providing 


information for four types of decisions. The four corresponding 


categories of decisions advanced are respectively, planning decisions, 
Structuring decisions, amplementing decisions and recycling decisions. 
Of the major theoretical constructs within educational evaluation, the 
CIPP model thus embodies a Significant reference to perspective within 
the model. 

Three of the six broad categories of purpose identified by 
Anderson and Ball (1978) deal with the type of decision served. They 
suggest that the purpose of evaluation may be to contribute to 
decisions about (a) program installation, (b) program continuation or 
expansion, or (c) program modification. The three other potential 
purposes of evaluation include (a) rallying support for a program, 

(b) rallying opposition to a program, or (c) contributing to the 
understanding of the basic process. 

Paralleling decision types somewhat, Rossi et al (1979) identify 
evaluations for program planning and development, monitoring, impact, 


ands COSesDenerie spULEPOSe's.. 


Methodological Orientation 

The emphasis placed on the type of methodology to be followed has 
in many cases had significant influence on the nature of a particular 
evaluation perspective. Some writers have even used the type of 
methodology as a basis for categorizing evaluative approaches. 
Wrightstone, in Essex (1969), identified six types of evaluation, 
namely, field orientated, laboratory studies, implementation studies, 
controlled experimentation, longitudinal studies, and case studies. 
The four major categories of evaluations advanced by Apple (1974), 


included the following: 
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a. anthropological approaches - using naturalistic models of 
BESeCarecheErom anenwopology ,; 

b. empirical methods - uSing research designs, regression 
analysis, statistical and measurement error analysis, 

c. secondary evaluations - uSing a central data bank, focusing on 
re-analysis of primary data, posing new questions of the same 
data, or combining multiple data sets, and 

d. process evaluations - using correlational techniques leading 
to causal conjecture from measures of home environment, 
parental pressures, expenditures, class size, teacher 
experience, etc. 

In an AERA monograph, Kraft et al (1974) offered the following 

“interdisciplinary"™ approaches to evaluation: 

a. the field study using anthropological techniques focusing on 
processes or transactions, 

b. the systems based economic or cost-value approach, 

@&e literary narrative approach, usually process oriented, and 

ad. the “portrayal” or overview approach which is a continuous and 
pervasive perspective such as the countenance model by Stake. 

A more research based categorization is used by Anderson and Ball 

(1975) who identify experimental, quasi-experimental, correlational, 
and survey types of evaluation methodologies. These methods are cross 
referenced with the different purposes of evaluations for a matrix 
approach to categorizing evaluations. 

Other methodological positions. Recent approaches to evaluation 

have focused on some new methods which recommend bases for a 


particular evaluative perspective. One of these is the adversary 


method which iS a quasi-legal approach to determining the value of a 
project or program or other educational unit. Other recent writers 
have emphasized methods which focus on the identification of quality 
as contrasted with quantity measures. Often these views are advanced 
as justification for subjective rather than objective evaluations. 
The view adopted in this study holds that quality can be described in 


quantitative terms. 


Accountability and Equality of Opportunity 

Evaluation as an attempt to demonstrate accountability gained 
importance during the 1970s. Lessinger (1970) pressed for what he 
called an "accomplishment audit" of our educational institutions in 
order to demonstrate the equality of our educational endeavors. In 
this case, the purpose of evaluation is focused on demonstrating 
productivity and efficiency. 

Often, an emphasis on accountability as a purpose for evaluation 
results in a narrowing of considerations to quantifiable elements, and 
productivity may become the ratio of outputs to inputs. Performance 
contracting, merit pay, and the voucher system have been advanced as 
ways of promoting accountability as they adopt the "market place" 
approach to the evaluation of an educational unit. 

In a Similar vein, the purpose of some educational evaluations 
has been to focus on inequalities of opportunity. Coleman's early 
report (1966) is now considered a classic example of such an emphasis. 


The primary purpose of evaluation from this perspective would be the 


elucidation of the equality of educational opportunity. 
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Political Nature 

One recent author draws attention to the inherent "political" 
nature of the evaluation as an important descriptor of any particular 
evaluaerverapproach. An evaluation can either be bureaucratic, 
autocratic, or democratic according to MacDonald (1976). He points 
out the potential problems with the way in which evaluation may be 
viewed by those involved. Emphasis is placed on the positive 
involvement within the evaluation of the participants in the 


educational system. 


Role of Judgement 

Should the evaluator pass judgement, or is this the domain of the 
decision maker or policy maker? 

EiCwanisVc EE cOmr oma DOvVemGUe St ron mom SL Nash eOanl tamlmeGe Se let oing wehie 
perspective of a particular evaluation. Some authors say "no", the 
evaluation should only provide the relevant information; others 
suggest that the essence of evaluation is in placing "value" and that 
this involves passing judgement; while a third mid-range position is 
that the evaluation should identify alternative positions from which 
the decision maker could choose in making the judgement. 

In summary, the foregoing evaluative concepts have been reviewed 
£Eom the Literature in onder to provide a basis for presenting the 
Conceptual framework adopted for this thesis: iIn-addition to these 
evaluative concepts a variety of approaches or strategies, froma 
number of disciplines each involved in evaluation, are briefly 
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evaluation perspective and a proposed model for monitoring, utilizing 
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the concepts and approaches reviewed here. 


Complementary Evaluation Approaches 

Program monitoring at the macro level focuses on gathering 
evaluative information about on-going programs. As such, program 
monitoring complements other, less macroscopic, evaluative approaches. 
Each of these approaches or strategies is referred to in this section. 

The literature relating to two of the above approaches, 
curriculum evaluation and social indicators, has contributed most to 
the development of the conceptual framework and model for this thesis. 
MHeseeake EeVIeCwea  anesome derais. 

The gathering of information concerning quality (the value of 
outcomes and the effectiveness of transactions) of the public school 
program in general and of major program components has been approached 
ENROUgGhHFavVaricty of perspectives. advanced by different related fields 
OESStUuady.. 

Educational planners propose and develop new pavcattonas program 
components based either implicitly or explicitly on evaluations of the 
on-going program (needs assessment). Organizational effectiveness 
analysts examine schools, with a primary view to evaluating the 
organization, but in so doing also evaluate the programs within the 
SGnool Ss CUCEICULUM CValUators are Anvolved in activities whach 
include gathering evidence concerning the value of specific curriculum 
materials, evaluations of a specified subject area curriculum, or 
ethnographic evaluations of particular operations. Information system 
specialists develop information storage and retrieval systems not only 


to facilitate the on-going school operations but also to provide 
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summary data of an evaluative nature. International and national 
developers of quality of life indicators include indicators on the 
quality of education. Analysts of organizational decision making 
processes emphasize the need for information, and propose methods for 
gathering such required information. Educational researchers conduct 
detailed inquiries into questions of quality, based on personal 
interests, institutional needs, or public pressures. 

Each of the above groups has developed different "models" and 
approaches. Different terminology, the overlap of activities, and 
unclear conceptualizations of the interrelationships among these 
fields of study have perhaps retarded the development of evaluation as 
SaeeleldnOmeSeuGy ean TeSMOWwnerrTghntr. 

However, the perspective of each of these fields of study is 
important to the monitoring of educational systems and on-going 
programs because of the complementary role each of them has with the 
monitoring function. Firstly, monitoring of on-going programs, being 
of a general nature, identifies the need for further study through one 
or more of the above disciplines. Therefore, the nature of the 
information produced would reflect the needs of each of these interest 
groups. Secondly, these fields of study often include within their 
evaluations, information gathered at the macro level which would be 


useful within a monitoring system. 


Curriculum Evaluation Models 
While evaluation models, as they have been applied, almost always 
focus on a. specified segment of a school system over a specified time 


period as in the case of projects, they nevertheless provide a useful 
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conceptual guide for meeting the needs of educational decision makers 
at the general level. The generally accepted reason for any 
evaluation model is to provide a framework which can assist the 
evaluator in identifying, analyzing, organizing and reporting data 
gathered for judgemental purposes. Of the evaluation models 
developed, that advanced by Stake (1967) has most influenced the 
development of the conceptual framework for the present study. 

Stake model. The model proposed by Stake (1967) is presented in 
Figure 1. The model essentially separates data into six categories. 
In one dimension data are separated into three categories, (a) the 
antecedents or the "givens", or that which precedes the system's 
operations, (b) the transactions or the descriptions of what exactly 
takes place during the operations, and (c) the outcomes of the system. 


On the second dimension, Stake suggests separating data into two 


categories: (a) intended or that which "Should be" and (b) observed or 


that which “is! or ‘was. 

Stake's model also suggests the development over time of 
Standards of excellence, or indicators that could be used for 
comparative purposes if the model were used in a recurring 
application. The role of judgements is also displayed. 

The CIPP model. “As mentioned earlier, the PDK evaluation model 
advanced by Stufflebeam (1971) emphasized evaluation for decision 
making. A type of systems model, four categories of decisions were 
suggested as being sequential during the life of an educational 
project. Planning decisions dealing with intended ends were to be 
served by context evaluation. Structuring decisions dealing with 


intended means, were to be provided information by input evaluation. 
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Figure 1 


A REPRESENTATION OF THE PROCESS OF JUDGING 
THE MERIT OF AN EDUCATIONAL PROGRAM: 
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Source: Collett, D. J. Curriculum Evaluation Project. 
Edmonton: Alberta Education, 1978. 


Process evaluation provided information for implementing decisions in 
order to utilize, control and refine procedures. The fourth type of 
evaluation was of a Summative nature providing judgements and 


reactions to the actual attainment of ends and providing for recycling 


decisions. 
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che Seesceonce model. Provus (1971) has previously been 
referenced for his’ emphasis onegoals vand.objectives. . The model 
advanced by Provus however, focuses on the identification of the 
difference between goals and achievement of those goals. The main 
function of evaluation is to identify the discrepancies between what 
was intended and results obtained. 

Contribution. While the model proposed by Stake presents a good 
overview and identifies interrelationships of components which should 
be addressed, the operationalization of this model has proven very 
irae re he On cm sca SOnmele BOC mmnAom NOt DCenmagdODEcCdm cOmamGiscare 
extent may be because of its apparent emphasis on "hard'' data. The 
NOdel men CLerOLemNascmuOGtsoCeCnm Appia CeNGureCELlyelne tneepReSent Study. 
However, the categorizations of data into antecedents, transactions, 
and outcomes has been adopted. In addition, the concept of 
establishing standards of excellence and the general system wide 
overview suggested a the Stake model have contributed to the 
development of this study. In the terms advanced by the CIPP model, 
this study focuses on the recycling decision, but views the type of 
evaluation as formative rather than summative when applied to an 
on-going school system. While focusing on goals, objectives and other 
outcomes, this study adopts the view of March (1972), referred to 
| earlier, namely that goals and objectives should be judged on what is 
currently believed important rather than the approach suggested by the 


discrepancy model of Provus. 


Social Indicators 


Work on the development of social indicators, especially that 
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related to education, has most notably been carried out by the 
Economic Council of Canada (Greenberg, 1974), O.E.C.D. (Carr-Hill and 
Magnussen, 1973), the U.S. Department of Health, Education and Welfare 
(Cobern, Salem and Mashbin, 1972), and by the National Centre of 
Educational Statistics (1975). Each of these has developed different 
sets of indicators, partly because of the various levels at which they 
function. For example, the 0O.E.C.Dé is interested primarily in 
comparisons on an international basis, while the Economic Council of 
Canada and the U.S. Department of Health, Education and Welfare are 
interested in data for a given country. The interests in the present 
Study are focused on indicators which provide information for the 
provincial and school district levels of administration. 

Aims of social indicators. In an attempt to aid in the 
understanding of social indicators, the O.E.C.D. (1976) has suggested 
that regardless of the specific perspective, all indicators have the 
following aims: 

a. to provide guidelines for the development of measures 
to fill the gaps in existing knowledge about 
SOCTLOS CC OHOMUCE CONG ELON S: 

De ro Link = Vanious Measures So as EO provide a picture note 
only of the relevant. phenonema, but also of the 
relationships between them; and 

c, to reduce information overload through concentration on 
relevant indicators and supporting data (p. 11). 

Characteristics of social indicators. Henderson (1974), 
Carr-Hill and Magnussen (1973), Andrews and Withey (1976), and Park 
and Seidman (1978) have all reviewed numerous definitions of social 
indicators which collectively include the following characteristics: 


a. concepts central to the generation of an information system, 


b. measurements of the performance of a social system, 


c. relevancy to policy, showing long-term trends and unusually 
Sharp fluctuation rates, 

ad. "time-series" data or statistics which can be monitored over 

time, 

€. quantitative and qualitative information of a limited yet 

comprehensive nature, 

f. descriptors of goal achievement which can be disaggregated to 

Snow, thes socral distrspbucion of these vachievements, and 

g. diagnostic of the components of the social processes within 

the system. 

The fewer indicators there are without missing important 
information, the better. Parsimony of indicators for the particular 
system level being referenced is important. As the Economic Council 
of Canada has stated in its Eleventh Annual Review (1974), provision 
should be made for 

...certain summary or principal social indicators (largely 

output-orientated) that would synthesize and represent the 

overall situation with respect to a given area, taking 

external interactions into account (p. 13). 
phremvtewoO1nty SUgGeScCS Various! tevVeEls Of Specificity of andicators, 
focused on various levels of curriculum or various institutional 
levels, yet with a view to combining sets of indicators into 
"principal" indicators for a more general monitoring report. 

In education, many of our important outcome variables are not 
measureable until many years after the student-clients have left 
school, but the evaluative information is needed as soon as possible. 


Therefore, where possible, current indicators of future outcomes are 


required. This may be achieved through the identification of 


7 A 1 % | a ¥ 
ips at st, el 


: 
2 ne 


30 


relationships, if any, that exist between measures of quality taken 
internal to the system and those longer range measures which must be 
taken external to the school system. 

Steps in developing social indicators. The 0.E.C.D. (1976) lists 
guiding principles in the development of Social indicators including: 
validity, disaggregation and aggregation, inter-temporal 
comparability, understandability, availability of data, and Jinks and 
overlaps. A more succinct approach to the development of social 
indicators is outlined by Henderson (1974) who identifies five steps: 

dementia ryethessyecen and =sub=sySstems. 

b. identify and taxonomize the outputs of the system, 

c. determine input and process factors which have a real 

influence on outputs, 

ad. determine the impact of specific programs and policies on 

outputs, and 

e. determine the relative importance of the outputs. 

Contribution. The aims identified in the social indicator 
literature have contributed to the perspective for monitoring an 
educational system, which is presented later in this chapter. The 
Characteristics referred to have influenced the type of variables 
selected and the way in which they have been handled. The steps 
proposed for developing social indicators have contributed to the 
establishment of the various stages within the model presented later. 

In addition to the contributions from evaluation models and the 
social indicator literature, the disciplines of educational planning 
and of organizational effectiveness also have contributed 


Significantly to the development of the model for monitoring an 


educational system. Each of these areas are concerned with 
evaluations of a more in-depth nature than proposed by the monitoring 
function. Each however have complementary roles and are reviewed in 


the following paragraphs. 


Educational Planning 

Literature in the field of educational planning has dealt 
extensively with evaluation. 

Beeby (1969) edited a collection of writings on the "qualitative" 
aspects of educational planning. Significant reference is given ina 
number of articles in this collection, to the development and use of 
indicators of adequacy and quality, both direct and indirect, that 
would be of use to planners. 

Kaufman, in his writings on educational planning (1972 and 1979) 
dealt extensively with the need and role of indicators of goal 
achievement and needs assessment in planning. The view of Kaufman is 
that any variable that can be shown to be representative of a later 
goal achievement can be considered an indicator. The position that is 
adopted in the present study differs somewhat from Kaufman's view on 
the nature of social indicators. It follows more closely the 
influence of the social indicator literature referred to earlier, that 
indicators should be goal referenced statistics or, in Stake's terms, 
outcomes. 

The present study has many of the characteristics of a needs 
assessment. The main difference is the macro perspective adopted 
here. The purpose of the model outlined later in this chapter is to 


provide the equivalent of an early warning system which could be used 
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to identify areas requiring more in-depth needs assessments. 


Organizational Effectiveness 

The emphasis, within organiZational theory, on different types of 
organizations and the particular characteristics of schools, have 
relevance to program monitoring. Thompson (1967) has categorized 
schools as organizations with ambiguous outcomes and incomplete 
beliefs about cause and effect. In a similar vein, Perrow (1970) 
refers to schools as "non-routine" with unanalyzable problems and many 
exceptions. While both Thompson and Perrow, along with Katz and 
Kahn (1966), used their analyses of organizations to support an 
evaluative approach based on identifying the characteristics of 
"healthy" organizations, they did point to the need to gather data 
from reference groups when evaluating organizations such as schools. 
This is consistent with Blau and Scott's (1962) identification of the 
Client as the prime beneficiary in service organizations such as 
schools and therefore the focal point of evaluative information. 
These concepts are accepted as partial justification for the emphasis 
in this ere client evaluative feedback via the followup method. 

Recently, Miles (1980) outlined an ecological model of 
organizational effectiveness which "...emphasises the importance of 
institutionaliZing a mechanism within the organization to monitor 
organizational effectiveness over time" (p. 385). He points out that 
evaluative information must focus on both goals and processes, 
gathering information primarily from referent groups. 

Contribution. The monitoring function adopts the emphasis of 


organizational effectiveness theorists on the client when evaluating 


human service organizations. Ina limited sense, compared to the more 
in depth organizational effectiveness evaluations such as those 
proposed by Mott (1972) or Steers (1977), the monitoring system 
FOCUSES On the Productivity or Goal optimization functions’ as outlined 


by these authors. 


Information Systems = 

Program monitoring approaches should be closely articulated with, 
draw from, and contribute to existing data banks. Many of the systems 
models display the importance of developing data banks and using these 
for evaluative purposes. 


The following section outlines in more detail the systems and 


evaluation framework which provides structure to this study. 


A Systems Evaluation Perspective: at the Macro Level 
In approaching the development of a monitoring model, the 
literature referenced has contributed to a conceptual framework of: 
a. the nature of the phenomenon to be evaluated (the educational 
system), as well as 
b. the nature of the process of evaluation. 
Generally, the perspective adopted iS a systems model of the 
educational activity and a macro level of monitoring the quality of 


education. This section presents particular systems and evaluation 


perspectives. 


A Perspective of the Educational System 


Through the influence of evaluation models such as those of Stake 


and Stufflebeam and systems theorists such as Van Gigch (1978), a 
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systems view of education has been adopted. In this view, teachers, 
curricula, and facilities with certain antecedent characteristics are 
brought together with students who have their own antecedent 
Charactervstires, resulting ino transactions within the school producing 
certain educational outcomes for the student. 

However, different organizational levels exist upon which to 
focus the evaluation. Further, the transactions within the school are 
too varied to take the "black box" approach to evaluation. Gathering 
evaluative information about high school students with the assumption 
that they have all had similar experiences (transactions) is 
erroneous. Therefore the systems approach adopted, draws specific 
aeeention to Ehe organizational dimension as weli as to the 
specification of the transactions from the students' experience, or 
what may be termed a curricular dimension. 

Organizational dimension of evaluation. The evaluation 
perspective changes depending upon the organizational level which is 
the focus of the evaluation. For example, Hastings (1969) identified 
the various levels of evaluation as being (a) teacher, (b) local 
school, (c) local system, (d) state system, and (e) national project. 

Forehand (1971) has drawn attention to just two levels of 
evaluation. The first, project evaluation iS viewed as formative 
evaluation and takes place within the process of curriculum 
development. A second level of evaluation is seen by Forehand as a 
diagnostic system, establishing standards and identifying regions 
where attention is needed. He describes institutional evaluation as 


tt 


considering: --e-the achievements of any particular program in 


relation to a network of other programs and goals, a function that 


Cannot be performed at the project level" (p. 582). The comments of 
Forehand are particularly noteworthy when it is considered that most 
of the models for educational evaluation that have been developed, 
have in one way or another been associated with evaluations for 
various nationally sponsored educational projects. 

Approaches not differentiated by institutional level, have been 
identified earlier by Guba (1969) as one of the failures of 
educational evaluation. He pointed out that the evaluators' 
traditional point of focus has been microscopic and that as we have 
shifted our perspective toward a more macroscopic level, the 
techniques, instruments, and procedures have often proven to be 
inappropiate. 

Qhe curricular dimension in evaluation. While the educational 
unit under evaluation may be an organizational unit, the particular 
program or curricular unit focused on will also determine to a large 
extent the nature of the evaluation envisaged. 

Various state assessments (e.g. Michigan Educational Asessment 
Program) seemed to accept gradevas the program level. in these 
evaluations, the unit»sof learning is the total experience for the 
year. The perspective could be more microscopic and consider for 
example, the contribution of a specific subject, or a more macroscopic 
view could be adopted considering for example, the contribution of the 
entire public school program aS a unit. One problem with the 
evaluation segment of the PPBES pilot project in Alberta arose because 
the particular program definitions chosen had no correspondence to an 
identifiable learning activity in the schools as experienced by 


students. While it was possible to assign costs according to the 
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particular "programs" identified, it was impossible to identify 
benefits from "programs" which in many cases were theoretical 


CONSErUCtES for the purpose of cost accounting. 


An Evaluation Perspective 

The monitoring of on-going school programs focuses on policies, 
programs and institutions of the educational system. dus cae at 
the system level hasS aS itsS main role the processing of information 
for the policy maker and the educational planner providing the basis 
for gaining support for the educational program and contributing to 
the understanding of the process. The evaluation of the educational 
system is air eerene ated by Johnstone (1978) from either the 
evaluation Of educational Curnpicula, educational projects, or the 
evaluation of individuals. Johnstone identifies two main purposes for 
system evaluation: 

a. current status analysis, and 

b. diagnosis and recommending decisions. 
The monitoring role adopted within the present study focuses on the 
current status analysis purpose. AS Suggested by Johnstone, a heavy 
emphasis has been placed on the use of indicators in filling this 
role, which can "provide a gestalt of a system's operation and 
development" (p. 255). Indicators are not variables measuring gross 
quantity but should rather be one of: (a) rates, (b) percentages, (c) 
averages, (d) differences, or (e) ratios which either aid, or contain 
within, comparisons of one quantity to another, or to a base quantity. 

The macro perspective proposed adopts the following positions 


with respect to the concepts identified earlier. 
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Goal orientation. Heavy emphasis is placed on goals and 
objectives which form the basis of provincial curriculum 
leadership. Goals and objectives move into the public domain 
facilitating the public documentation of the achievements of the 
public school system. Goals are evaluated in terms of present 
views of what should be. 

Decision types. Collecting evaluative information in a format 
Suited to the target decision maker iS important. The macro 
level of decision making is recognized. This includes the 
public, who provide financial and other support for educational 
programs. The monitoring function focuses on recycling 
decisions, decisions of continuance or modification in the terms 
of Stufflebeam (1971) and Anderson and Ball (1978). 
Methodological orientation. Primary data sources for program 
monitoring iS envisaged as existing school files and periodic 
Sampling of ex-clients of schools through a longitudinal design. 
Program monitoring must include a recurring inductive inquiry 
phase. 

Accountability and equality of opportunity. A major purpose of 
program monitoring is to open the priorities and achievements of 
the schools to the public. In so doing, information is provided 
which allows for accountability in terms of goals, objectives, 
and other outcomes and the presence or absence of equal 
opportunity may be demonstrated. 

Role of judgement. The main function of monitoring is the 
provision of information for top level policy makers, curriculum 


planners and the public. 


14. 
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Organizational level. Data are collected at the provincial level 
and disaggregated to the next two major levels of decision 


Making, namely the school districts or counties, and the schools. 


Program. In monitoring the effects of schooling, continuing 


emphasis must be placed on distinguishing more clearly the 
"program'' or specific school experiences that students have 
undergone in reaching their respective achievements. Care must 
be taken to differentiate programs "offered' by schools and 
programs "experienced" by students. The former often are 
representative only of administrative units within the school. 
Categorization OmiVvantables seyinespnocess cfitechanging cobilected 
variables into evaluative information is aided by appropriate 
categorizations of the variables. A loose adherence to systems 
models, such as that of Robert Stake, can aid the evaluator not 
only in the identification of data that should be collected but 
also in the way it should be analyzed and disaggregated in order 


to be more meaningful. 


Needs assessment. Educational system monitoring of programs iS a 


type of on-going needs assessment at the macro level, but does 
not replace the more in-depth needs assessment that may be 
required from time to time. 

Organizational effectiveness. Public institutions, which hold a 
monopoly on the service provided yet depend heavily on general 
public support, present a different milieu for effectiveness 
ratings than most other organiZations. Monitoring systems must 
gather feedback from the clients of such institutions. 


Information systems. Due to the need for quick turn around on 


large volumes of data, program monitoring must have computer 

based information systems as an integral part. 

The preceding position statements, regarding the various 
evaluative concepts and strategies that have been identified from the 
literature, provide the framework within which the following model is 


presented. 


A Model for Monitoring an Educational System at the Macro Level 
Educational programs in Canada are constitutionally a provincial 
Pesponsipilaty. AS a result, the provision of free public education 
(K-12) has been a joint venture involving both local school boards and 
Ene Departement Of Hducation in each Canadian province. Politicians 
Bueno saALOmc nce Cue Nem © Lense CWOM MIS tisellelOnale MeVveLsS™ would 
be the users of evaluations at the proposed macro level. 
paneer GecisitOnemakerseakt the macro devel have to deal with the 
allocation of finite public funds to the growing demands of social 
programs, they require comparative information concerning the current 
status of the quality of each social program. Comparisons have to be 
made with alternative educational programs as well as with other 
competing social programs. Questions often posed include: (a) what is 
the quality of education in a utilitarian sense? or, (b) how much is 
being received for the expenditures made? 
A model designed for use in monitoring an educational system, its 
sub-systems and on-going programs, was developed for the present 
study. In developing the model, use was made of the criteria 


identified earlier for monitoring at the macro level. The model 


served as the framework for the present study and appears in Figure 2. 
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Three major phases are identified in the model: the data collection 
phase, the inductive inquiry phase and the information phase. In 
addition, three stages are identified in each of the last two phases. 
Hea cesecarch SuDcpYroblemseor ether chests paraklel these six stages. 
Data Collection and Preparation Phase 

Data collection and preparation is an important first step in the 
model." The typessof data’ which showld be collected sare describedsin 
earlier sections of this chapter. The proposed emphasis is on data 
directly related to the student-client of the school. Due to the long 
range impact of schooling, follow-up data must be an integral part of 
a monitoring system, along with readily available data from existing 
Peke sr 

While not included in the present research, information gathered 
from other referent groups such as parents and teachers is also 
important. Qualitative information of the ethnographic type could 
also be collected. However, as the goal of monitoring iS Summary type 
information, detailed qualitative information would not normally be 
oneluded: 
Inductive Inquiry Phase 

This phase focuses on the analysis of the data collected for the 
purposes of identifying inherent meaning. The three stages of the 
inductive inquiry phase are: 

a. identify outcome indicators of educational quality, 

b. identify antecedent and transactional variables (both 

student-based and school-based) which account for the variance 
in the quality indicators, and 


c. determine the relative explanatory power of the variables 


4) 
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identified in the second stage. 

The results of these three stages are the identification of indicators 
of quality education and the identification of possible contributory 
variables. The purpose for identifying these possible contributory 
Variables is to permit the disaggregation of the quality indicators in 
order to examine in more detail the distribution of quality within the 
educational system: Hence the main products of this phase are quality 
indicators and variables to be used for the purpose of disaggregating 
ENeCRQuaigie yr nawCcaAcORnsre 

This phase was developed in recognition of the crude nature of 
macro evaluation, due to the influences of contaminating variables and 
the level of development of appropriate methodologies. The inductive 
inquiry phase seeks to assure the re-examination of the quality 
indicators selected as disaggregating variables in each cycle of 
monitoring. 

Information Phase 

After the identification of the quality indicators and the 
selection of antecedent and transactional variables as disaggregative 
Maigrabvesn che presentation of the data in an informative and 
evaluative manner is desirable and becomes the focus of the third and 
last phase of the model. 

The first stage involves the development of provincial norms as a 
type of operationally based standard, for each of the quality 
indicators. In the second stage, the focus iS On examining the 
distribution of "quality" among different groups of student-clients. 
In the third stage, the data are examined from the perspective of 


Gqifrerent: 1nsticutional units within the school system. 


Complementary Evaluations 

The model displays the relationship of the monitoring process 
With other complementary evaluations. 

The inductive inquiry phase has the potential for raising 
questions requiring experimental research aS a followup, as certain 
antecedents or transactions are identified as being associated with a 
higher quality of education. More detailed research may be initiated 
through analysis of the particular quality indicators identified with 
a view toward development of alternate sources of data. 

The first stage of the information phase would contribute 
PHOViNnCcLTal educational anag@icators fer potential inclusion within the 
DEOadeuearcanOrmscOClal Ina TCAacOuSsOneeneRqualveyror Lares eMore 
in-depth curriculum evaluations may be suggested as a result of the 
level of quality indicated within each of the subject areas of the 


school program generally. 


The second stage has the potential for identifying differences in 


the quality Of education received by certain sub-groups of students. 
Such information has the potential for identifying the need for 
CuLrEercullumeevValuarion studies co determine causes of and: possible 
remedies to inequitable situations. 

The third stage of the information phase, when carried out ona 
province-wide basis year by year, could identify organizational units 
which should possibly be candidates for more in-depth organizational 


effectiveness evaluation. 
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Summary 

This chapter has briefly outlined a research perspective which 
emphasizes on-going interaction with the data, a view which recognizes 
the need for hypotheses and new directions to grow out of the data. 

A variety of concepts within evaluation generally which help to 
identify a particular conceptualization of what evaluation is and 
should be, have been reviewed. Examples of literature from different 
disciplines which are concerned with evaluation and whose activities 
often meet and interface at the macro level of evaluation have been 
reviewed. The approaches that have been identified have had a major 
influence in the development of this thesis. 

The final section of this chapter has provided a description of 
the evaluative perspective adopted for this study along with an 
outline of a model for monitoring an educational system. The model 
presented has guided this research and provided the structure for this 


thesis. 
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CHAPTER 3 


REVIEW OF THE CURRICULUM EVALUATION PROJECT 


The Curriculum Evaluation Project (Collett, 1978) was 
commissioned by Alberta Education in 1970 to start the development of 
a broad, synoptic approach to the evaluation of schooling in terms of 
province-wide goals, objectives, programs, priorities and issues. It 
was a pilot test of what could become an on-going systematic approach 
to gathering and organizing evaluative information about the high 
school curriculum from a provincial perspective. The CEP was 
undertaken primarily as a result of expressed concerns over the 
apparent higher cost of providing vocational programs compared with 
other more traditional programs in the high school. 

Whatesthesantenes ofp CEP was) to. collectuantormation from alt 
sources, the pilot test of the approach focused on information 
associated with and provided by the student-client. 

Department of Education records (student records and school 
A-Cards) were the source of province-wide educational information 
about these students. Student questionnaires provided individual 
self-report information and evaluative feedback on goals, subject area 
objectives, school services and learning activities. The project was 
longitudinal in design, with three student questionnaires being 
completed: (a) the first, during the grade 11 year (1971), (b) the 
second, one year after the grade 12 year (1973), and (c) the third, 


five years after the grade 12 year (1977). 
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The population selected was the entire grade 11 student body 
enrolled in the 1970-71 school term in the province of Alberta. Since 
background information would be required on each student, all students 
for whom there was not a complete grade 9 and 10 record were deleted. 
A random sample of the remaining 30,000 plus students was drawn, 
giving a sample size of 2754. 

For the sample identified, 2224 of the first questionnaires Or 
80% were returned. 

The second questionnaire was, through error, mailed only to the 
2224 persons who had responded to the first questionnaire. Of these, 
1340 questionnaires were returned for a 49% return (60% of those 
mailed). 

The third questionnaire was mailed to the entire sample of 2754, 
of which 740 (27%) were returned. 

Broader scope. As the Curriculum Evaluation Project developed, 
the need for an on-going systematic approach to gathering and 
organizing '"macro-evaluative" information became more apparent. Hence 
a broader, though implicit, purpose of CEP became the demonstration of 
ways in which such an on-going approach might function. Data 
collected and methods used often went beyond the initial purpose in 
©Fder GOrmMectsthis additional yobzective: 

Data gathered focused on goals and goal statements, student 
programs actually experienced, subject area objectives, school 
services and generalized learning activities. The student-clients 
involved were asked to evaluate "how adequately did your high school 
program meet these aims for you?" Accompanying educational and 


occupational achievements were also recorded. Evaluative information 
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regarding what was their experience was supplemented on the third 
Questionnaire by soliciting their views regarding what should be in 
the future. 

More specifically, the CEP data bank contained the following 
types of provincial information: 

a. antecedent information about the student prior to enrollment 

ine eheshaghe schooler 

b. the results of student-client feedback on the process of 
course and program selection, 

c. the results of computer analysis of student records 
identifying course patterns or "programs" experienced by 
Students, 

d. self-report information regarding career plans and (through 
the followup questionnaires), career achievements, both 
educationally and occupationally, 

e. student-client evaluation of the school's achievement of 
general goal statements and subject area objectives, and 

f. student-client ratings of the importance of general goal 
statements and subject area objectives. 

Sources and type of information were as presented in Table 1, 
while a complete listing of the variables within the data bank is 
provided in Appendix A. A copy of the segments of the followup 
questionnaire which are relevant to this present study is provided in 
Appendix B. 

The CEP report did not make available district or school level 
categorizations of information gathered. Hence, the report presented 


findings for the province as a whole rather than laying the basis for 


(a) Departmental Student Records 


(b) 


(e) 


(a) 


(e) 
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TABLE 1 


SOURCES AND TYPE OF INFORMATION 


- observed antecedents: 


MOUSER VCOmEEan Sat rons. 


- observed outcomes: 


Departmental School Records 
- observed antecedents: 


First Student Questionnaire 
- observed antecedents: 


—- observed transactions: 


junior high ability and 
achievement scores, 
student age, sex; 


student programs; 
educational 


achievement, 
etc. 


grades 


EGeai eS orrerned by School, 


enrollments, 

type of school, type of 
cerms, region, number of 
teachers. 

(01)-1971 


father's education, 
mother's education, 
educational and 

occupational plans; 


teaching methods. 


Second Student Questionnaire (Q2)-1973 


—- observed transactions: 


— observed outcomes: 


Third Student Questionnaire 
- observed outcomes: 


— intended outcomes: 


teaching methods; 


educational and 
occupational 
achievements. 


(93)-1977 

educational and 
occupational 
achievements, goal and 
objective achievement 
ratings; 


importance ratings on 
goals and objectives. 
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an on-going program of monitoring at the district or school levels. 


Data Preparation Methods Used by CEP 

The prepararion that was undertaken for many of the variables are 
evident £rom the descriptions of the variables themselves. A few of 
the variables in the data bank were derived from rather complex 
analysis of student records. Two of these derived variables require 
some special explanation. These two were: 

Categorization of high school achievement. Five levels of high 
school achievement were identified as: matriculation, high school 
diploma, grade i1 entrance to technical programs, completed grade 10, 
and an incompleted grade 10. ‘Criteria for each of these levels were 
EnCnetdcne th ed and! 1Seed. Ae COMPuEech Program was wertren) cO analyze 
the high school record of each student and assign an ordinal number 
from 1 to 5 to each student file. For certain analyses thiS variable 
was treated as having an ordinal scale. 

Student program. This variable identified the type of student 
program in terms of the group of courses taken by the student. 
Categories were determined in two ways. The first categorization 
included the traditional matriculation, business, vocational and 
general groupings. An analysis of student records by clustering 
techniques, supplemented by subjective judgement, resulted ina 
further breakdown of these groupings into 15 categories. It was then 
necessary to establish mutually exclusive yet exhaustive criteria for 
each of these categories and to write a computer program to analyZe 


the student records and assign category designations to each file. 


Grade 11 registrations were used for this purpose. The detailed 
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description of the criteria for each category iS available in the 


Curriculum Evaluation Project report (Collett, 1978). 


Summary 

The data bank established for the Curriculum Evaluation Project 
provided the data for the present study. This was a longitudinal 
Study OLreche grade Ii class of 1970-71. sThe present Study selected 
some of the variables used by CEP, but utilized additional data that 
were in the data bank but not used by CEP. Variables that were 


selected are described in the next chapter. 
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CHAPTER 4 


METHODOLOGY 


This chapter outlines the various methodologies used in 
undertaking the research. The conceptual framework outlined in the 
preceding chapter identified three phases in the monitoring of 
on-going school programs. The first phase of the monitoring process 
outlined in Chapter 2 (Data Collection and Preparation) was, in part, 
completed before the present study was undertaken, as the CEP data 
bank (referred to earlier in Chapter 3) provided the variables adopted 
for this study. It must be noted however, that the report of the CEP 
did not include an analysis of all of the data that had been collected 
at that time. SA reviewsofr the methodologies associated with the data 
collection and preparation phase is provided in Chapter 3. 

Phase one: data selection and preparation.» While the complete 
application of the model presented earlier would include the 
collection of data, this present study required only the selection of 
data from the CEP data bank and appropriate preparation of the data in 
the development of required variables. The criteria used for the 
selection of appropriate variables and the methods used in the 
development of additional variables from the raw statistics within the 
Gata bank are outlined in the first section of this chapter. The 
report of this activity provides the listing of the variables utilized 
in the present study. 

Phase two: inductive inguiry. The inductive inquiry phase of the 


present study encompasses the first three of the six research 
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sub-problems posed in Chapter 1. These are: 

Ae to identify general and subject area indicators of quality 
education; 

Bis to identify student-based and school-based variables which 
account for significant variance in the indicators of quality 
education; 

She to determine, for selected indicators of quality education, the 
total variance which is accounted for by the student-based and 
school-based variables and the explanatory power of each of these 
variables. 

Phase three: informational. The third phase of the monitoring 
process proposed is the information phase and includes, in the present 
Sceudy, che following sub-=probiems: 

4, BO VCOMNPUTG PLOVincial norms for each sandicator of quality 
education; 

lore to identify, for selected indicators of quality education, 
differences among groups of students classified on the basis of 
student programs and student characteristics; 

6. to develop profiles for selected schools and districts using the 


various indicators of quality education. 


Data Selection and Preparation Phase 
This section deals with the methodologies used in the data 
preparation of the variables selected. Methods followed by the 
Curriculum Evaluation Project have been reviewed in Chapter 3. 


Additianal new variables were prepared from the data bank specifically 


for the present study. The methods associated with this additional 


data preparation are outlined in this section. 


Variables Selected from the CEP Data Bank 

As a first step, variables used by CEP were examined and those 
meeting the requirements of the conceptual framework were selected. 
Many of the items within CEP were either of a Survey nature or dealt 
with the establishment of importance ‘ratings of goals and objectives 
and therefore not considered appropriate for the on-going monitoring 
function addressed herein. 

Secondly, an examination of the complete data bank that had been 
assembled for CEP yielded much additional data which were relevant to 
the monitoring function. The variables pertaining to subject areas 
were especially important. 

Table 2 presents in brief form the variables selected for use in 
the thesis, showing their type, description, and source. Variables 
are also identified as to whether they were used in the CEP report, 
Calculated so as to Summarize CEP data, or taken from the original 
data bank. A detailed listing of the selected variables is presented 
in Appendix C. 

While many of the variables selected came directly from either 
Alberta Education files or the student questionnaires, others were the 
results of preparatory analyses which are referred to briefly in the 


following paragraphs. 


Methods Used in Preparation of Additional Variables 


The detail of the methodology used in the preparation of 


variables through the calculation of means, or the tallying of various 
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entries in the data bank is not elaborated here. It is hoped that 
these methods are apparent aS a result of the labelling of the 
Variables BFurther explanation is required in thencase: of fourcof the 
variables. 

Student program. The CEP had categorized the student programs 
two different ways, one into four categories, and the other into 15 
categories. It was felt however that categories more discriminating 
than four but more parsimonious than 15 could be developed. Asa 
result the 15 categories were collapsed into nine and a new variable 
developed to provide this designation. Analysis in the Inquiry Phase 
Showed which of these groupings would be the most appropriate. 

BehOGlEdTSstr eeecone. Thee student irecondskcontainedereference 
only to the school attended. It was necessary therefore to establish 
a catalogue of school districts with their respective schools. A 
computer program was then developed to assign the appropriate district 
code to each student's record based on the school code. 

Career status. The Follow-up questionnaire was in two parts, 
dealing with occupational status and type of employment. For the 
purposes of the present study, these two variables were collapsed into 
one. 

Achievement of goals. The CEP followup questionnaire gathered 
ratings*on 34 sub-goais as listed in the Department of Education 
Program of Studies (1977). In the interests of parsimony, these 
sub-goals were grouped under their original goal headings and mean 
achievement ratings were calculated for each of the 12 major goals of 


education. 


BS 
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Inductive Inquiry Phase 

The inductive inquiry phase, undertaken during each cycle of the 
monitoring activity, examines the variables used in order to provide 
the evaluator with information helpful in designing possible follow-up 
research and with information on ways of improving or extending the 
.collection of data during the subsequent cycle. The objective is the 
continual testing and upgrading of the variables used in the 
monitoring system. This phase encompasses the first three 
Sub-problems of the study. 

Implied in the theoretical model developed in Chapter 2 is that 
each of these steps would be required each time a monitoring study was 
undertaken. The purpose of repeating this phase each time would be to 
confirm or call into question the results of prior monitoring and to 
introduce new variables as they are developed and added to the system. 

The following are descriptions of the methods and procedures used 


an addressing ehe chree sub jproblems associated with this phase. 


Identification of Quality Indicators 


Sub-problem 1: to identify general and subject area 
indicators of quality education. 


Views differ as to whether quality indicators should include only 
measures of outcomes or whether measures of certain "process" or 
transactional variables should also be included. According to the 
latter view, such transactional variables would be included as quality 
indicators, where they were known to be either desirable educational 
events or could be shown to be associated with desired outcomes. 


The position adopted for this’ study was to consider outcomes 
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AHONesaASMINGUCaACORS Ofaquality. = A Iast Of Criteria for such quality 

indicators, was presented in Chapter 3. 
Procedures. The steps used in identifying the quality indicators 

to be incorporated into the monitoring system were as follows: 

ile. Variables were selected from the data bank which, according to 
earlier identified criteria, none considered to be outcome 
measures. Average ratings were developed for the provincial 
goals by computing means for the respondent ratings of the 
relevant sub-goals. 

ais Parsimony of outcome variables was sought by a procedure which 
combined highly intercorrelated variables or dropped one variable 
EPOM pales On highly anterconrelated’ Variables. USpeciiacaLily, 
this procedure involved intercorrelating all the outcome 
variables selected and then examining critically the correlation 
coefficients. Where high correlations existed one of the two 
related variables was dropped if theoretical overlap was 
apparent. However, if the two variables were thought to be 
conceptually discrete, a combination of the two variables into 


one was undertaken. 


Identification of Disaggregative Variables 
Sub-problem 2: to identify student—based and 
school-based variables which account for significant variance 
in the indicators of quality education. 
In identifying independent variables most explanatory of variance 
in the quality indicators, important antecedent or transactional 


variables were identified for further examination in the information 


phase of program monitoring. If a significant amount of the variance 
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for a given quality indicator was found to be accounted for by an 
antecedent variable, such as the sex of the respondent, it is 
important that the nature of the relationship between the indicator 
and the particular variable be shown. For example, whether males or 
females had higher scores on a given quality indicator might reveal 
that one or the other of these two had been the recipients of a higher 
quality education as measured by the indicator involved. Such an 
antecedent variable would then be used to disaggregate the quality 
TNatCaeore nn OLGer elLomscnowe thes nacvces of (this relacionship. 

The assumptions of the statistical procedures used require that 
Such predictor variables be mutually exclusive and non-interactive. 

Procedure. The procedure followed for identifying variables 
accounting for variance in the quality indicators was a combination of 
an interative clustering algorithm and one-way analysis of variance. 
This procedure (SEARCH) was developed by Sonquist (1973) and is 
available as a computer program within the OSIRIS IV Statistical 
Analysis and Data Management System (1979). For each dependent 
Variable, SEARCH sequentially divides the sample on the independent 
variable with the highest explanatory power (ANOVA), subsequently 
resulting in clusters of the sample with specified characteristics 
based on the splits made on the various independent variables. A 
one-way analysis of variance is computed for the final groups in order 
to show the amount of variance accounted for by the resulting model. 
Examination of the path of the various splits can identify interactive 
terms thus permitting the formulation of combined terms which contain 


the interactive elements. 
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SEARCH runs. One SEARCH run was made for each quality indicator, 
using each as a dependent variable. The results were then examined 
for interactions and the independent variables most frequently chosen 
for clustering were selected as potential disaggregate variables. 
These potential disaggregators were further validated by the multiple 


regression approach used in sub-problem 3. 


Relative Explanatory Power 
Sub-problem 3: to determine, for selected indicators of 
quality education, the total variance which is accounted for 

by the student-—based and school-based variables and the 

explanatory power of each of these variables. 

The determination of the explanatory power of the independent 
variables was undertaken primarily to show: 

a. the possible extent to which appropriate variables have been 

identified for disaggregation of the quality indicators, 

b. the comparative explanatory power from one quality indicator 

to the next, and 

c. the more influential (marginal) dependent variables for each 

Quai cye narecaeoly, 
The findings of a, b, and c above, help to validate the disaggregate 
variables, show the importance of various antecedent and transactional 
variables, and identify potential research projects. 

The methodology considered most appropriate for the third 
sub-problem was multiple linear regression. The major assumptions 
underlying multiple linear regression analysis could be met, since the 
initial analysis undertaken using the SEARCH methodology had reduced 


the number of variables and identified interactions. Since much of 


the data was'of a nominal nature, a specially adapted regression 
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analysis developed by Baker et al (1972) was used. This adapted 
analysis, labeled Multiple Classification Analysis (MCA), uses a dummy 
variable approach in order to accommodate the nominal variables. 

The task therefore was to run an MCA analysis for each quality 
indicator uSing the independent variables identified in sub-problem 2. 
From these analyses regression equations were formulated. Multiple 
COErelativon cocHiicrvents were also callculated=in order to show the 


degree of association in each case. 


Information Phase 
The third phase of the monitoring scheme and the second major 
part of the present research took the basic variables dealt with in 
phase-two and presented them in three different ways: 
a. provincial norms on the quality indicators, 
b. a disaggregation of the indicators based on student programs 
and student characteristics, and 
Cc. a disaggregation of the indicators based on school 
Characteristics. 
The following are the outlines of the methodologies used for each of 


these sub-problems. 


Provincial Norms 


Sub-problem 4: to compute provincial norms for each 
tnd Caton Onequallity education. 


Provincial norms for each of the quality indicators provide 
aggregate information concerning levels of achievement for all of the 


outcome areas represented. 
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The methodology used in determining these provincial norms was a 
combination of means and standard deviations along with categorical 
breakdowns where appropriate. Means were used for quality indicators 
based on variables which used interval scales, categorical breakdowns 
were used for quality indicators based on variables which used nominal 
scales. 

For selected goal indicators based on items which had five 
Category Likert-type interval scales, percentages of the sample in 
Various categories were computed and the top two response categories 


Soimapscadg forse sinplaecrcyeineaarashand lang. 


Student Characteristics Disaggregation 
SUS DSODLeIe > mcOnmlachernywa ms OMmmsclecrodmindrcarers. Of 
quality education, differences among groups of students 
classified on the basis of student programs and student 
characteristics. 

Disaggregations of the quality indicators based on student 
Characteristics was undertaken to show the distribution of the general 
quality of education among certain student groups or, in other words, 
to show the variation in achievement toward the quality indicators by 
various student groups. 

The variables selected as disSaggregators had earlier shown 
informative potential in that they were associated with certain levels 
of variance in the quality indicators. 

Student-—based predictor variables fell into two major categories, 
namely antecedent and transactional. Antecedent variables focused on 
background characteristics of an educational, social or individual 


nature. These variables included age, sex, ability, previous 


scholastic achievement, self ability ratings, socio-economic status 
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and career plans. Only one student-—based transactional variable was 
included. This variable identified the cluster of courses which 
comprised the "program" that the student followed while in school. 

The procedure. Computer-based two dimensional cross tabulation 
was used to identify differences among groups classified on the basis 
of the various personal and school program variables. Since the. 
student program variable was identified most often as the variable 
associated with the greatest amount of variance in the educational 
quality indicators, a three-dimensional cross tabulation was 
undertaken with categories of selected indicators, selected student 
Characteristics, and the various student.programs. 

Percentage distributions were selected as the most appropriate 
way of showing the varying loci of achievements on the quality 
indicators by students of varying backgrounds and following various 


high school programs. 


Institutional Disaggregation 
sub-problem 6: to develop profiles for selected schools 

and districts using the various indicators of quality 

education. 

Institution—-based variables were also considered important within 
the monitoring system. Those institution-based variables which were 
descriptive of various types of scnools or various school 
Characteristics were used, as well as identifying variables of 
specific schools and specific school districts. 

Type or characteristic variables represent features of the school 


or school program which resulted from certain policy decisions. The 


school and school district disaggregate variables are important as 
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they identify the different administrative or decision making units 
within the province. 

Cross tabulation procedures were again used in developing 
institutional profiles of achievement on selected outcome quality 
indicators. As the number of schools and districts were large and the 
purpose of this part of the study was illustrative only, a small 
number of representative schools and districts were selected for 
example. Schools and districts were selected representative of urban 


and rural origins aS well as of the different regions of the province. 


Summary 

The methodologies used within the framework of the proposed model 
for monitoring school programs have been presented in this chapter. 

In dealing with sub-problem 1, the identification of quality 
Bnercacors, methods wsed ancluded antuitive examination of underlying 
concepts, correlations, and various statistical processes for 
combining variables. Sub-problem 2 required the use of the 
computer-based SEARCH process, followed by analysis of variance, in 
ene teentitication of both antecedent and transactional variables 
which were most associated with variance in the quality indicators. 
Multiple classification analysis, a specialized analytical program 
Similar to step-wise multiple regression, provided the ability to deal 
with sub-problem 3, which was the determination of the extent to which 
variance was accounted for by the model and the relative explanatory 
power of the most Significant independent variables. 

Sub-problem 4, the calculation of provincial norms on the quality 


indicators, required the simple calculation of means where 


appropriate, and in other cases the determination of categorical 
percentages. Disaggregation based on student characteristics and 
institutional levels (sub-problems 5 and 6) required the application 
of statistical breakdown procedures and cross tabulations along with 
appropiate filter systems. 

Ald of the’ statistical procedures wsed were provided by the 
OSIRIS IV Statistical Analysis and Data Management System developed at 
the Institute for Social Research at the Universtiy of Michigan. 

The results of the study described herein are presented in the 


next 2 chapters. 
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CHAPTER 5 


RESULTS: INDUCTIVE INQUIRY PHASE 


This chapter presents the findings for the first three 
Sub-problems which form the inductive inquiry phase of the study. The 
results for the remaining sub-problems (the information phase) are 
contained in Chapter 6. The first Sub-problem dealt with the 
identification of indicators of quality education. The second 
Sub-problem dealt with the identification of student-based and 
school-based variables accounting for Significant variance in the 
MmncaCatoOnsS OLequal teymeducation, = sslbsproblem 3 sought to compute the 
total variance accounted for by all, and the explanatory power of 
each, of the student-based and school-based variables, in association 


With selected indicators of quality education. 


Quality Indicators 


Sub-problem 1: to identify general and subject area 
indicators of quality education. 


Only variables which were classified as "outcomes" were 
considered for use as potential indicators of quality education. Two 
categories of outcome variables existed. The first category was 
broader in scope than the second and focused on general educational 
outcomes. The second category focused on outcomes specific to subject 
areas. The primary objective at this stage was to seek parsimony of 
indicators by minimiZing overlap, yet assuring reference to a cross 


section of goal areas and theoretical constructs. 
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General Quality Indicators 

Measures of educational outcomes gathered while students were 
Still in school, along with measures of career achievements and later 
evaluations of educational goals by the former students were selected 
from Ehe Gata bank fom furEehern analysis. —~Usang these outcome 
measures, answers to three questions were sought: 

t What was the extent of overlap identifiable conceptually as well 
as statistically? 

ane Which measures could be combined in order to reach the objective 
of parsimony? 

=e What numerical representations for each outcome measure would 
clearly enable comparative evaluations? 

The outcome measures taken from the data bank along with the resultant 

quality indicators are presented in Table 3. The results for each 

research question follow and deal in turn with internal measures, 

Career achievements, and goal ratings. 

Entcernal measures. Intercorrelationsein= the range Of .63 to -86 
were recorded among the three outcome measures: (1) the level of high 
school achievement, (2) the average mark over all courses, and (3) the 
total number of high school credits achieved. The highest correlation 
was found to exist between the variables "high school achievement" and 
"total credits'’. One of the numerous criteria of "high school 
achievement'' was the number credits earned, thus contributing to this 
high correlation. Since the "total credits'' variable did not include 
diploma criteria nor entrance requirements to post-Secondary programs 


and, given the high correlation with high school achievement, it was 


66 


67 


wAZasJeS “sseujzl4 
“UZ LESH, ‘e}enbsepe AusA/ajyenbepy jUSsdued -ZI¢¥ 


wSLLENXS UOLPeOLUNWWOD, 
:e@zenbepe AusA/e enbspy jUSsoUedg -| 14 


,UOLPEWUOSUT 
Bsf, :spenbepe Ausn/e enbspy USdUed -Ol¥ 


,Sserousz,edwoyg 
o1seg, :s}enbspe AuaA/az,enbsepy USdUSsd -G¥# 


4J8SUBO YIM palysizeEsS AUSBA/PSl4Sl3eS JUSedUdd -BH 
OOO'ZIL$ USAO BulLuues jJUSdUaeq -L# 

LSAel snzeis gol 4ue},uog-O8UlLdg SHeusay -9¢ 

gol e Buryses pue peAo,dweun },USdUeq -GF 
UOLJeEONPS USeyzZuNy JO Sues SHeueay -p#¥ 


SyyueW LOOYUDS YALU 40 SHeuaay -E#¥ 
UuOL}PELNOLUZeW BHuLASslyoe JUSd.Usd -ZH 
BWwOL|diLp ,ooyos YHiy BulAstyoe jJuSsdUueg -} 4 


PeALuag SUO}VeODLpUT AiLLeNnH | eueuaey 


sbed Yxsu uo peut zUaD 


weuBoud sseuzyis LeotsAud poob e usiiqeisqZ “9 

sao1j}oeud 
u}.Leey LeyUSeW pue ,eotsAUud pooh 40 BulLpueysuepun ue do|sersq °q 
S}lqey YyyLeey |eucSued puNnos Ysi{qe}syA ‘ke 


:Ayeses pue Sseuzyts “YLLBPSY JO SeSpP!t SY} PUBSUSpUN PUe BdlL}9RUdq “EL 
AL @AL}POSLJS SHulL_Lsay pue seapt Buryeotunwwoo ut Ajyittqe do,eraaqg «9 
Susy,O 4O UOLZPEOLUNWWOD BY} BuLpUue-SuspuUN UL LLLHS doLlseasqg -q 
sebenbue, Us}}LUumM Pue [LeuO UL LL LXS do|lseasg ‘e 
>UOLZLOLUMWWOD UL SL LLHS Goterseq “Zt 
eHpe,tmouy Jo sisAjeue 
pue sO Zinsund oy} UL SpOoYy VeW OLjgtzZUSLOS Ajdde 0. ALL iqge do,erseg °95 
S8tuobezyeo {NyHuLuesew O}FUL UOLJPeWUOsUL SZLULHUO 0} AL{LLqe do|lsansq *q 
A{,eo16o, Butpsecsoud pue Bulryuty 4$O SLLLHS AdoLeAsg ‘e 
>ueuueW BAL LOSelgqo 
pue [| BOijtyud e@ UL UOLPeWUOJUL |BSN pue ‘azA,eue ‘SeziueGBuo OF MOY UUeaZ ‘41 
SOlJLLLqQe pue S}ySeue,UuL LeLosds dojsarsg ‘a 
Ss}deoucd pue UOL}eWUOFUL 4O PUNY e do|sAeg ‘q 
SSOUSLOS [SLOOS pue [eot}pewseyujyew ‘Saeouelios 
{eun}zeu ‘Suequnu $0 BSN Sy} UL SLLLHXS PUue HutpueySuspun dojsasq ‘e 
:S@eLousz,edwoo shps,mMouy ,eLroeds pue s{1tyS OLSeq do|ansq ‘of 
>(LOOYUOS dezy¥e SueeA BAL) UOLZPeEONpPA 4O SLeBOH ZT} SY JO SHul1zZey - | eUuse}Xxy 
UOLJOSSSLYES Uedued 40 Bsaubaq 6 
yaeyHOeUq Shem 8 
uoLzednos0 40 BuLiyueu uszpUdg-Os8Ultd L 
Auobezyed | euotz,edns5g “9 
SNES PUSsWAQ! dwy “G 
UuOL}PeONPS YwauyyUNS JO Suead v 
>{OOUDS ue}Je SUuReA BAL PSe}UOdsu se S}USWeASLYOe useueng - [| ebUUA}XKA 
peretyoe S}Lpaud [oouds YHLy 4o uaqunu |e OL Se 
SeSUunoo [Le 4waAO yYUueuW SHeusaAy Cre 
SOUBULUS SNFLFZSUL LBOLUYoSs} uo ‘ewo,dip 
LOOUDS YHty ‘UOL}PeELNOL4uzZeW “"So'L ‘}PUSeWAaAeLYDSe LOoYos YHLy yo [ersq Pai 
>LOOUDS ULYZLM S}PUBWAaASLYOY - | euUdue UT 


yUuUeg BEG SUL WOUY pPa}yOeUyXA SSuNSeepy aewodjiNO | eueuUEy 


——o eee 8 eee 


SHOLVOTIGNI ALIIVAO 1IV4YSN39 G3ATYSG 3HL GNV SAYNSVAW 3WOOLNO IWYAN|AD 


Srl olicd Via 


eee acer 


1 44OM SUOIS{O9ep UeeUeD O} Pa Yelau 
$O P[4OmM, tSa}penbaepe AusA/ay,enbaepy j.UedUaed -OZ#H seotAues Bul, Lesunood pue uol;}JeWUoOsJUL BSN OF ALL'Ge eu doleraag “gq 
LOOYDS YHtLYy 4uwazPJe YUOM J4O PL YOM |BUY OF DESeq S{L}yHS Gotenaq ‘e 
>{ooyuos YyBiy uezse YYOM JO PL YOM BY }PNOGe UueSay ‘42 
wnSLLtLAS sadunoseu uewny pue (eroueuly ‘| eunzeu BHuyGbeuew ut sttt7s dojernseq -q 
yUSWSHDeUeW, *‘oe}penbsape Auad /ayenbsapy }US9DU8d -6hL4 SOetztLtqrsuodseau 
pue saidioutud Otwouode jo BHBulpueysuspun ue dojsarag ‘e 
“SLLLHS PUSweHeuew dolensq ‘O% 
,84NZFLND S}ue ([ed}}Oeud pue sBuly ay} 
38}eioguddy, ‘s}enbepe AueA/a}enbsepy jUueduaeg -gI¥4 Burpn,out erpew snotuea YBnouy}y UoLSsSeudxe-4y[es SAL}eaeud doltaaneq ‘q 
Ssw4uO4s SNOLUeA UL AjYNeSq 4OJS UOLZPE}Seudde aeAIZIND ‘e 
>PLyOmM 94} UL AjYNeAaq Ppue BUN [ND |aPeLoauddy 6} 
awit} Buns}e| 
$O ASN HulAys}}eS pue Sstim eB OC} Pes, [LEM YOLYUM S}SeuazUt doLeraq “q 
youl | BAL}BSUD pue [LeNnPOS, Se ut ‘ LedtsAud -~ SHLZLA}VOe awit} Sunstsat 
B4unsts], :e}enbsepe AusA/azyenbsepy jpUSedUed -LIF $O eGueu e Ut UO} }PEMLOLZUed PueMoOY |BpN}i1}7e BALZ1SOd e doljereg ‘e 
:eWL} SuNnstel SSN O}F MOY Uueaqy “gt 
SL[SASBL [ePuUOlL}PeUUSs}UL pue 
Leuotz}eu je eHe,lLyuey pue Aji}yUuept [Leun}y{[No so Buy lees e doltensg ‘a 
dtyusuezZt}zt!9 p_,yuom pue ueltpeueg 
$O SOLZFLLLQtsuodssu pue uo1}e611qo ay} 40 BulpuezysSuaepun ue dolsreg ‘p 
SOLYLEL'EQ}Suodsdu pue s}UBiu OLALO yO SSeUBaUeMe Ue do|eAeg ‘95 
,dtysuezizyt9 yUBWUUBAOCB 
pooy, te}yenbepe AueA/azyenbsepy ~Usd4eqd -91¥ $O SWUOJ YSBYYO PUe UeLPeUReD BY} JO BuLpUeYSuspuN Ue dolsreq *q 
Ayusdoud e}yeAtud pue O1,1qnd wos }OedSeu 40 |BpN}1}}e US dojansg ‘e 
:U@ZtXL9D Pooh e eq 0} UUueAay ‘ZI 
AyeLoos 
uetpeueg $0 Spuewsp Bulrbueyo ay} 0} YSNfpe oO ALtELtqe do,sereg ‘9a 
yUBWUOULAUS |, eoLSAYd pue 1eLs0s 
BurBueyo e 0} YSNfpe 0} AELILGe ey Pue yo SSauUdUueMe Ue doO|[sAag “q 


,sebuey) GUNN BY FESoW OF AXE LIGe B4y 
pue}ySispuy,, s‘e}yenbepe AusA/a}enbsepy jUSdUsg -Gi¥ pue }uaSeud ayy YUYIM Ast}USpP!L ‘}ySed BY JO BuLpUueY-SuepUN do|serneg ‘ze 
>PLUOM 
eu} Ut} Bdeid eye} FEU} SSHUCYD BY} PUeYSUSpUN 0} Au} PUB YNOGe UueAaq “94 
,OuLuues) Buruuea, Buoltejt |, wos SSouueHea pue A,}SO}UND [Len}OS, [a zUL do,srneg ‘q 
S4utseg, ‘e}enbepe AuseA/aj,enbepy j.Usd4Ueq -HIL# Butuuea| Puemo} Sepnzyti}}e BALY}SOd e do{sraq ‘e 


:Bupuuee, uo} BSutsep e dolserneq ‘Gh 
SUOL}NZLLFSUL LeYSLOOS SNO}LUeA 4O SJPUBWSASB!YSe 
pue Set}vittqpsuodsdau ‘suoltzoUuNnsy 4o Buppueysusepun ue doj,srseg °*9 
seo1oyo | eucosusad 
Bur yew UeyM SUBYYO JO SEN|LBA BY }JUNODOeS O}UL Bye} OF URS “q 
1 S498440 SLeENPLA}PUt 
498dsey, ‘a}yenbepe AusA/aj,enbaepy ~UeD4adqd -EL#H $O AZPLUBLP PUue YZUOM BY} YUOJF PFOSBdSSeUu PUB UOL}eLOeudde dojeasq ‘e 
:S@LAIS Osh 
pue Sjot,eq BurAuean jo atdoad yuytm Buole eG 0} Pue YOedSeu 0} UUReSa ‘PHI 


(penut}yUuos) € A1aVL 


. 4 
ae a vd 
\ 
2 — 
, < 5 ° _ 
1 
> - 


in _ 
_ e 


ri 


wih 


r 
5 
4 


le 
amid. ber 


af 


; 


; a er 
t1 Gite %o amicfiv get sepesos aeet sees 
+ : ee > 


dropped as an indicator. No combination of measures was considered in 
this category. 

The mean of the high school achievement variable could have been 
used aS a quality indicator. However, the decision was made to focus 
attention on the percentage of students that had achieved a high 
school! diploma. In order to accomplish this, the percentage of 
Students in the top two categories of/ this variable were selected as a 
general indicator of quality education. 

The average of all high school marks was retained as a general 
indicator of quality education because of the lower intercorrelation 
Of this variable. 

Career achievements. Six different career achievement variables 
were considered as shown in Table 3. Intercorrelations among these 
measures were quite low, in the range of .16 to .42, with the 
exception of employment status and the Pineo-Porter ranking of 
occupations for which the correlation coefficient was .71. The low 
correlations were considered imateatave of little overlap and 
therefore supported the inclusion of each variable as separate 
measures of career achievement outcomes of the educational experience. 
The relatively high correlation in the case of the one pair of 
variables (r=.71) suggested that both were assessing similar outcomes. 
Comparison of the two items revealed that this indeed was the case, 
with one being a categorization of job type and the other, the 
Pineo-Porter ranking, being a combination of job type, prestige 
ratings and salary. However, one category of the employment status 
variable, unemployment, was not covered by the Pineo-Porter scale. 


The result was the identification of the five quality indicators as 
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shown in the table. In the case of the general quality indicators 5 
and 7 (column 2 of Table 3), the mean was considered appropriate in 
that the units of measure were years. Indicators 6, 8 and 9 were more 
categorical and therefore percentages were chosen as the unit of 
measure. 

Goal ratings. The 34 goal statements ranked by the respondents 
who participated in the C.E.P. followup study (listed under each of 
the 12 respective goals in Table 3) had been administered in two sets 
of 17 goal statements, with each set ranked by half of the 
respondents. As a result, correlations and factor analysis could only 
be done for each set of 17 statements separately. 

Correlations and factor analyses for the two sets of goal 
Statements did however show some statistical justification for 
combining the rankings of the goal statements. This finding, plus the 
fact that the goal statements were identified by the Department of 
Education as expansions on its 12 goals of education, were considered 
justification for combining the ratings of the 34 goal statements 
under each of the 12 goals of education. 

The original 34 goal statements are presented in Table 3 grouped 
under the 12 major goals numbered 10 through 21 inclusive. After 
combining the goal ratings to form.12 variables, intercorrelations 
among these variables were computed and the resulting correlation 
coefficients were still small, ranging from 0 to .35. Four response 
categories had been provided for each statement, "very adequate," 
"adequate," "not adequate," and "don't know." For the purposes of 
developing a comparative indicator of quality, the two categories 


entitled "adequate" and "very adequate" were combined. The quality 
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indicator derived therefore was the percentage of respondents judging 
the corresponding goals to be either adequately or very adequately 


achieved within the high school program in which they were enrolled. 


subject Area Ouality Indicators 

Outcome measures specific to each subject area were considered 
important as indicators of strengths and weaknesses in the school 
offerings.’ Such indicators could help to identify curricular areas 
that may be in need of more detailed evaluation. Measures contained 
in the data bank and selected for further analysis included the number 
of credits attained in each subject area, the marks achieved in each 
Subject area, and respondent ratings of the extent to which the 
subject area objectives were met for them. The outcome measures 
selected along with the derived subject area quality indicators are 
presented in Table 4. The same questions used in identifying the 
general quality indicators were again posed in identifying subject 
area indicators of quality. The statistical analyses undertaken 
identified a number of internal and a number of external indicators. 

Internal indicators. Conceptually, the sins of credits 
achieved was a measure of the extent to which students participated in 
course offerings within each subject area, while the average marks 
acCtatneagwerera measure of themlievelsof their achievement in the 
subject area. Pearson product-moment correlations between these two 
measures yielded low coefficients for most subject areas, supporting 
the concept that each of these waS measuring a different outcome area. 


For the general academic, fine arts, physical education, business 
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education, home economics and industrial education courses the 
correlation coefficients’ ranged from’.02> to 237." For “matriculation 
type" subjects, coefficients were higher, in the range of .46 to .71. 
Nevertheless, correlation coefficients were considered low enough 
overall to treat these two as separate measures. 

Focusing on the extent of participation, three possible 
indicators were selected. Indices were computed for each. First, the 
mean of the number of credits taken would be useful in showing how 
much time the average student spent in each subject area while in high 
school. Second, the extent of participation could be shown by 
computing the percentage of students who had taken one or more courses 
in each subject area. The third indicator selected focused only on 
the students who had taken at least one course. To indicate the 
extent of participation by this group, the average number of courses 
taken was computed. 

Focusing on the level of achievement of students in each subject 
area, three further indicators were selected and indices for each 
computed. First, the average mark achieved by students in each 
subject area was adopted as one indicator. Second, in order to 
provide a clear indication of the distribution of marks, particularly 
in terms of the way that they might influence further study by 
students, the proportion of A and B grades combined was used as 
another indicator of subject area achievement. Third, since students 
do not achieve credits with a grade of F, and even with a grade of D 
cannot proceed to the next level of studies, the percentage of grades 
D and F combined was taken as a further indicator of achievement (or 


lack of) °in each subject area. 
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External indicators. The measurements available from the data 
bank included a Likert scale rating of the objectives of each subject 
area by those who had taken at least one course. As part of the 
Curriculum Evaluation Project, respondents had been asked to answer 
only if they had taken at least one course in the specific subject 
area. Further checks based on the indicators referred to in the 
preceding paragraph showed that in three subject areas (fine arts 
subjects) a large number had responded who had not actually taken one 
or more such courses, and these were rejected. A cluster analysis was 
undertaken for the responses associated with the entire set of subject 
objectives. Relatively high intercorrelation coefficients (varying 
from subject area to subject area, but generally in the range of .50 
to .70) for the objectives associated with each subject area resulted 
in clusters of relatively high correlation coefficients falling almost 
entirely within subject areas. These results suggested the combining 
Sfethese-ratings antojone indicator for each subject area.  -Thisswas 
done by using the arithmetic mean as shown in Table 4. 

Intercorrelations were computed between the mean rating and each 
of the credits taken and the average mark, in order to determine the 
extent of overlap. Positive correlation coefficients in the range of 
-21 to .47 resulted when the number of credits was intercorrelated 
with the mean rating for all subjects except English, Social Studies, 
Social Science and General Mathematics. In these four subject areas, 
the size of the coefficient was very low for the relationship between 
the number of credits taken and the rating on the objectives. Low 
correlation coefficients resulted when the average marks attained were 


correlated to the mean rating on the objectives. 
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Sub-problem 2: to identify student-based and 

school-based variables which account for significant variance 

in the indicators of quality education. 

The second sub-problem was to identify from the student-—based and 
school-based (antecedent and transactional) variables, those 
accounting for the greatest amount of variance in the quality 
indicators identified in the previous section. The variables 
accounting for the greatest variance would then be considered for 
further analysis and for use as disaggregating variables in 
determining the various levels of quality of education received by 
qifferent groups of students. 

Analysis of variance was used to identify differences in the 
selected indicators of quality education for the antecedent and 
transactional variables within the data bank. Antecedent variables 
included student-based data such as sex, ability, past achievement, 
self ability ratings, parents' education and career plans, as well as 
school-based data such as the school region, urban or rural 
designation, the school code and the school district code. 
Transactional variables included a variable descriptive of the cluster 
of courses which comprised the students' high school program as well 
as various descriptive statistics of the school attended such as 
number of credits offered, pupil-teacher ratio, number of counsellors, 
academic or general or business or vocational programs offered, and 


the type of school term. 


Tt 


Significant Predictor Variables 

The results of the analysis of variance are presented in Table 5. 
The Eta-squared (proportion of variance explained) is shown for each 
predictor where it was larger than .01 with the level of significance 
Se Crom DO renO Gas cles. Oni. 

Examination of these results was undertaken to identify mutually 
exclusive predictor variables which would best explain variance in 
both general and subject area quality indicators for the purposes of 
further analysis. In the interest of brevity, only a few of the 
quality indicators were selected for detailed analysis. 

Student program. Based on Eta?, variable 54 (student program) 
was generally found to be significantly related to the greatest number 
of the dependent variables selected. For this reason, the student 
program variable was selected for subsequent analysis. 

School transactional variables. In only a few cases, were 
significant Eta*s found for the measures descriptive of school 
transactions, displayed in Table 5 as variables 55 and 58 through 61. 
Variables such as school size (v55), pupil-teacher ratio (v58), type 
of term (v61), and type of program (v60) were found to have either low 
Or no association with the indicators of quality education. Of these 
school-based variables, the pupil-teacher ratio (v58) was selected for 
further analysis since it showed the highest relationship with 
Variance in the quality indicators. Only one school-based variable 
(v58) was selected due to the relatively high intercorrelations among 


these variables. 
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Sex. Variable 63, sex of respondent, showed fairly pene teeene 
though low relationships with most of the dependent variables. No 
Significant Etas were found for career satisfaction nor for three of 
the subject areas (Social Studies, Science, Music) on the ratings of 
the objectives. 

Student ability. Scholastic Aptitude Test results, 

(v64-verbal, v65-quantitative,v66-total), as predictors for the 
selected quality indicators, accounted for variance in a range up to 
approximately 25%. Each of these predictors was intercorrelated and 
since variable 66 (T-SCAT) showed consistently higher results it was 
selected for further analysis. 

Past achievement. With few exceptions, grade nine achievement 
scores (v67-v73) showed predictability for each of the quality 
indicators, in a range slightly higher than the ability scores. 
Again, because of the high intercorrelations and the higher 
predictability of the grade nine average (v73), scores in the 
individual subject areas (v67-v72) were dropped in subsequent 
analyses, and only the average was included. 

Self-report antecedent variables. Of the antecedent data 
Supplied through student self reporting (v74 through vV78), student 
self ability rating (v74) and career plans (v78) showed the highest 
level of predictability. As with the ability and past achievement 
predictors, predictability was found over most of the dependent 
Variables, with career satisfaction and the rating of subject area 
objectives as exceptions. 

School antecedent variables. Neither the region of the province 


(v79) nor the urban-rural split was significantly associated with 
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variance in the quality indicators. School codes (v81) and toa 
lesser extent, district codes (v82) showed predictability of variance 
in the outcome measures shown. Since handling variables which have 
large numbers of categories is difficult and costly in multi-factor 
analysis, the school and district identifying variables were not 
selected. A number of schools and districts were however selected for 
comparative analysis in the information phase of the study which is 
reported in the next chapter. 

In summary, one-way analysis of variance was undertaken for each 
of the independent variables (v54,v55 and v58-v82) using selected 
indicators of quality education as the dependent variables. 

Generally, the predictability of the independent variables was similar 
from one dependent variable to the next. Based on the extent of 
variance in the outcome measures accounted for as evidenced by Eta? 
Shown in Table 5, seven variables accounting for the greatest amount 
of variance were selected for further analysis. These seven were 
student program (v54), pupil-teacher ratio (v58), sex of respondent 
(v63), T-SCAT ability score (v66), grade nine average (v73), self 


ability rating (v74), and career plans (v78). 
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Sub-problem 3: to determine, for selected indicators of 
quality education, the total variance which is accounted for 
by the student-based and school-based variables and the 
explanatory power of each of these variables. 
The third sub-problem had, as its objectives, the determination 


of the extent to which the student-based and school-based variables as 


a group, accounted for variance in the quality indicators; the 
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identification of differences in the explanatory power from one 
indicator to the next; and the discovery of the relative explanatory 
power of the various student-based and school-based (predictor) 
variables. 

A monitoring system of the type proposed in Chapter 2, ideally 
should include those independent variables which account for all of 
the variance in the indicators of quality education used. Therefore, 
it would be important to include within the monitoring system an 
on-going check of the extent to which the variables selected were 
achieving this ideal. In addition, should remedial action be 
necesSary, a ranking of those transactional and antecedent (predictor) 
variables which account for the greatest amount of variance in the 
outcome variables would be helpful in directing further AGEL ONE MO hecc 
least in providing direction for more in-depth evaluation. 

Because some of the predictor variables were nominal rather than 
continuous variables, the statistical analysis selected for this task 
was Multiple Classification Analysis, which is eciar to multiple 
regression analysis when using dummy variables. Two different 
approaches were used, one to determine the predictability of the total 
group of independent variables and the relative explanatory power of 
each of the predictors when the others were held constant and the 
second, to determine the marginal contribution to the total variance 
for each of the predictor variables. 

Multiple Classification Analysis (MCA) assumes an additive model 
with no interaction between predictor variables. With this 
statistical technique, the predictor variables may be of a nominal 


nature, while the dependent variable must be of the interval type. 


82 


Since this phase of the research was exploratory and intended merely 
to illustrate types of analyses that could be undertaken ina complete 
monitoring system analysis, it was delimited to a few general outcomes 


and a selected number of subject area outcomes. 


BOobechis analysis, the number of predictors was limited to 
student program, pupil-teacher ratio, sex, student ability (T-SCAT), 
and grade nine average. Two predictors, self ability ratings and 
student career plans, which had shown promise in the earlier analysis 
were dropped from the analysis since their use would have resulted in 
a decrease in the available "N"' which in turn could have introduced 
some bias. The results of the Multiple Classification Analyses are 
presented in Table 6. 

General outcomes. The highest degree of association, between the 
predictors and the general outcomes, was found for the two internal 
measures tested. Over 52% of the variance was. accounted for in the 
case of the "level of high school achievement," while in the 
prediction of the average high school mark, over 56% of the variance 
was accounted for by the five predictor variables. Of the three 
external outcomes selected for analysis (further education, 
Pineo-Porter ranking, and career satisfaction), the highest degree of 
association was found for the measurement of further education, where 
31% of the variance was accounted for. Very low predictability was 
found for the career satisfaction variable with only about 14 of the 


total variance accounted for. 
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The predictor variables with the highest predictability overall 
were the student program and the grade nine average. As the 
individual beta scores in Table 6 reveal, student program type was the 
highest predictor for the level of high school achievement. The grade 
nine average however was the best predictor for the average high 
school mark. The sex of the student showed higher relative 
predictability in the case of average high school marks than was the 
case with high school achievement. Of interest, in view of the 
earlier ANOVA results, was the relative low predictability discovered 
in the case of the ability rating as measured by the T-SCAT score. 

Both high school program type and the grade nine average had 
equal beta weights when predicting the number of years of further 
education. Sex was the variable which accounted for the greatest 
amount of variance in the Pineo-Porter occupational ranking, and 
accounted for a significant amount of the variance in the case of the 
further education outcome. 

Subject area outcomes. Two internal outcome measures and the 
rating of objectives were selected for analysis using four 
illustrative subject areas: Matriculation English, Social Shlidies: 
General Science, and Music. Of the 17 subject areas included, these 
four were selected as examples for the purpose of representing a 
matriculation subject, a core subject, a non-matriculation subject, 
and an optional subject respectively. The proportion of variance in 
these subject area outcomes explained by the five predictors is shown 
in the lower part of Table 6. The highest proportion of variance was 
explained in the case of the number of matriculation English credits 


taken (66%) and the lowest by the number of credits taken in Music 
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(4%). This reduction in the proportion of the variance explained, is 
mostly a result of the adjustment factor applied due to the smaller 
"N's" in the case of General Science and Music. 

The individual beta scores for each of the five predictors used 
reveal that high school program was highly predictive of the number of 
Credits taken by students in the various subject areas, while the 
grade nine average was highly predictive of the average marks received 


in these subject areas during high school. 


In attempting to determine the relative explanatory power of the 
respective predictors, the approach reported in the preceding 
paragraphs "controlled" the other variables by holding their influence 
constant. Andrews (1973) suggests an alternative approach which 
identifies the "marginal" contribution of each predictor variable. 
This procedure first computes a coefficient of multiple correlation 
with all the predictors involved and then a new coefficient with the 
specified predictor removed. By comparing the two coefficients, the 
Marginal contribution of the specific predictor can be determined. 
The result is usually reported in terms of the proportion of variance 
which is explained by the addition of the predictor and that 
Proportion of the variance which is not explained by the other 
predictors in the set. 

Following this procedure, marginal explanatory power for each of 
the five predictors was computed and is shown in Table 7 using the 


selected internal outcomes as dependent variables in turn. The 
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student program variable demonstrates the highest marginal explanatory 
power cee predicting the level of high school achievement (.19). It 
was also highest when predicting the number of credits achieved in 
each subject area. The grade nine average had the highest marginal 
explanatory power when predicting high school average (.08) and when 
predicting average marks in English and Social Studies (.07 and .06) 
but not in General Science (.04) where the student program was again 
the best predictor. 

The amount of variance accounted for marginally in the case of 
student program ranges from a low of four percent in the case of marks 
an General Science to a high of 23*percent in predicting credits 
earned in Social Studies. Nineteen percent of the variance in the 
dependent variable "high school achievement" is accounted for by the 
program followed by the student, over and above the predictability of 
the other variables. 

Table 7 shows that two percent of the variance in General Science 
(.02) and five percent of the variance in Matriculation English marks 
(.05) as well as three percent of the variance in the overall high 
school average (.03) is accounted for by the sex of the student. This 
ts .aemarqinal scontribution, in additionsto the contribution of other 
variables such as student ability or previous achievement scores. A 
very low marginal contribution to variance in the outcomes was found 
in the case of the T-SCAT ability rating (.005 or less). 

In summary, it was found that over 50% of the variance in high 
school achievement level and in the average high school mark was 
accounted for by the five predictors identified for analysis (Table 


6). Predictability for all three subject area outcomes (rating of 
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objectives, credits received, marks attained) varied from gis ig) Stig 
Matriculation English to a lowean Muss¢c.8 The approach of “hoiding the 
influence of the other variables constant and the marginal 
contribution approach, both demonstrated the significant degree of 
association between the program that the student follows in high 
school and the students' achievement of most of the internally 
measured outcomes. To a lesser extent, but more than for other 
predictors, the high school program was also found to be significantly 


associated with the externally measured outcomes. 


Summary 
This chapter has presented the results of the inductive inquiry 
phase, which encompassed the first three sub-problems of the study. 

Sub-problem 1: A total of 20 general indicators of quality 
education and seven subject area specific indicators were identified 
from the available data. 

Sub-problem 2: Of the 25 student-based and school-based predictor 
variables, seven were identified as accounting for the greatest amount 
of variance in the indicators of quality education. These included 
student program, pupil-teacher ratio, sex of respondent, T-SCAT 
ability score, grade nine average, self ability rating, and career 
plans. The first five were carried forward for further analysis. 

Sub-problem 3: Up to 56% of the variance was accounted for by the 
five predictor variables; this in the case of predicting the high 
school average mark. Overall, the high school program followed by the 
student accounted for the most variance in the outcome measures. 


Although predictability was lower for the externally measured 


88 


89 


outcomes, up to 30% of the variance was accounted for, even though 
these outcomes were gathered five years after the students had left 
schoo? 

The following chapter continues with the presentation of the 
findings of the study, focusing on the last three sub-problems which 
formed the informational phase of the monitoring model developed for. 


this study. 
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CHAPTER 6 


RESULTS: INFORMATIONAL PHASE 


The preceding chapter dealt with the first three sub-problems of 
the study. This chapter presents the results associated with the 
remaining three sub-problems. These sub-problems formed the 
Information Phase of the monitoring system proposed. 

Sub-problem 4 called for the computation of provincial norms for 
each of the indicators of quality education that are identified in the 
previous chapter. These norms, as well as having the potential for 
comparison at the provincial level at different points in time, serve 
as bench mark indicators for comparison with the level of educational 
quality experienced by various student sub-groups and the level of 
educational quality provided by various sub-systems within the 
province. The analysis associated with the fifth sub-problem compared 
achievement levels on the indicators of quality education for various 
student sub-groups based on student characteristics and student high 
school program Variables. These variables had previously been found 
to be significantly associated with variance in the educational 
outcomes. The sixth and final sub-problem required a similar 
disaggregation of the quality indicators according to school-based 
variables and displaying of the quality indicators for selected 
schools and school districts for illustrative purposes. 

While it has been shown in the previous chapter that each of the 
student-based and school-based variables was associated with a 


Significant amount of variance in the outcome measures as well as the 
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degree or extent of association, the detail of this involvement 
remains to be shown. For example, since the variable "sex of the 
respondent" was shown to be associated with variance in high school 
marks, the question which remains is: did the males or the females 


receive the higher grades? 


Provincial Norms for Quality Indicators 


Sub-problem 4: to compute provincial norms for each 
indicator of quality education. 


The indicators of quality education selected were listed in 
Table 3. In the "general" category are three indicators of quality 
education reflecting outcomes measured internally or within the 
school, five indicators measuring external outcomes of a career 
achievement nature, and external ratings of the 12 goals of education. 
The subject area indicators of quality education listed in Table 4 
included three indicators dealing with the extent of student 
participation in each subject area and three indicators dealing with 
the marks achieved and the distribution of these marks within each 
subject area. Objective ratings which were externally measured via 
the followup questionnaire were also included as indicators of quality 
education. 

The provincial norms that were calculated for both general and 


SUbVECENARCaANquaLLEyY Nar Cartons -LOlLow. 


Provincial Norms for General Indicators of Quality Education 
The norms based on the provincial sample, 2753 records in the 


case of the internal indicators and a 26% return of 710 in the case of 


the external indicators, are presenced an Table 8. Overall, 65.1% of 


TABLE 8 


PROVINCIAL NORMS FOR GENERAL QUALITY INDICATORS 


Indicators Norm 
~ > 

Internal: N= (2753) 
Percent achieving high school diploma Seis | 
Percent achieving matriculation status 5065 
Average of high school marks 61.6 
External: N= (710) 
Average years of further education the. V 
Percent unemployed and seeking a job Bats: 
Average Pineo-Porter job status level 46.5 
Percent earning over $12,000 Sylea 
Percent satisfied/very satisfied with career 64.4 
Overall goal' rating (%adeq./very adeq.) 55.3 

# 1 Basic Competencies (4adeq./very adeq.) 69.2 

# 2 Use Information (4adeq./very adeq.) 69.1 

# 3 Communication Skills (%adeq./very adeq.) 65.4 

# 4 Health, Fitness, Safety (%adeq./very adeq.) 60.5 

# 5 Respect Others (%adeq./very adeq.) Seo 

# 6 Desire Learning (4adeq./very adeq.) 59.2 

# 7 Understand Changes (%adeq./very adeq.) 56.1 

# 8 Good Citizenship (%adeq./very adeq.) 53.8 

# 9 Leisure Time (4adeq./very adeq.) Sins 
#10 Appreciate Culture (4adeq./very adeq.) 45.4 
#11 Management Skills (4Zadeq./very adeq.) 45.0 
#12 World of Work (%adeq./very adeq.) 40.6 


‘A complete listing and description of the goals of education is 
provided in Table 3. 
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the students had met the requirements of a high school diploma and 
30.5% of the sample had achieved matriculation in the sense of 
attaining entrance into a university program. The average of all high 
school marks was 61.6%. 

The norms for indicators relating to career achievement showed an 
average of 1.7 years of education beyond high school and 5.8% 
unemployed and seeking a job. For those employed, the Pineo-Porter 
job status level was 46.5 on the average with a range of 15 to 75, 
while 31.1% were earning $12,000 per year or more. Close to 65% of 
the respondents reported that they were either "satisfied" or "very 
Satisfied" with their career. 

On the adequacy of the 12 Goals of Education, the percentage of 
respondents who rated the sub-goals either "adequately" or "very 
adequately" achieved for them through their school program ranged from 
a low of 40% for goal #12 (preparation for the world of work) toa 
high of 69% for goal #1 (develop basic competencies). The development 
of basic skills and special knowledge competencies (#1); organizing, 
analyzing and using information (#2); and skills in communication 
(#3); all were rated adequately or very adequately achieved by 65% or 
more of the respondents. Three of the goals were rated at least 
adequately achieved by fewer than 50% of the respondents including: 
the appreciation of culture and beauty (#10); the management skills 
associated with economics, natural and human resources (#11); and the 
development of skills and information related to the world of work 
(#12). The six remaining goals ranged from 51% to 60% adequacy 
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Provincial norms were computed for six different internal subject 
area quality indicators for each of 17 different subject areas. These 
indicators, displayed in Table 9, show three different measures of the 
extent to which students were enrolled in each subject area and three 
different measures of the level of achievement by students in each 
Subject area. In addition, for those students who took courses in the 
respective subject areas, the percentage of students rating the 
various subject area objectives as having been either adequately or 
very adequately achieved for them, form an externally developed set of 
qualitey Gnadreators. 

Student participation in subject areas. The average number of 
credits taken by students in each subject area shows the highest 
Participation to have occurred in Matriculation Science (13.1), 
Matriculation English (11.4), Business Education(11.5) and Social 
Studies (10.4). Lowest participation was in the three fine arts areas 
with an average of approximately one credit per student in each area. 
The second indicator in Table 9, the percentage of students taking at 
least one course in the specific subject areas, takes a slightly 
different perspective on the degree of participation. Social Studies 
and Physical Education have the highest percentages (99.1% and 97.5%), 
being the only two specifically compulsory courses in the high school 
program. Slightly over 55% had taken at least one Second Language 
course and approximately 25% had taken courses in each of Home 
Economics (28.1%), Industrial Arts (26.2%) and Vocational Education 


(24.2%). In the fine arts areas almost twice as many students had 
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taken courses in Art as in Music (21% versus 11%), with Drama falling 
between the two (13%). When Mathematics is compared with Science, it 
is evident that a greater proportion of students participated in 
General Mathematics courses (51%) than in the General Science courses 
CST Le Conversely, a greater proportion of students participated in 
Matriculation Science (79%) than in Matriculation Mathematics (66%). 

For those students who had taken courses in each subject area, 
the third indicator shows the degree of involvement in the subject 
area. The measure used was average number of courses taken in the 
subject. The highest number appears in the Matriculation Sciences 
(5.3 courses) partly due to the large number of three credit or "half" 
courses in science rather than the five credit "full" courses common 
in other subject areas. On the average, students who were enrolled in ~ 
Matriculation English and those enrolled in Business Education took 
over three courses in the respective fields. 

Student achievement in subject areas. The assumption that the 
quality indicators dealing with student marks are entirely a measure 
of student achievement that can be compared from subject area to 
subject area cannot be justified, as at least to some extent the 
indicators may merely reflect different grading practices in the 
various subject areas. In either event, the comparison -of these 
indicators may raise questions for further study and at least provide 
a provincial standard for comparison within subject areas. 

As evident in Table 9, the average grades were highest in Music 
(73.3%) and lowest in the Mathematics and Science areas (58.8% to 
59.4%). With respect to letter grades (A, B, C, D, F), Music is again 


highest with 84.7% of the students receiving grades of either A or B 
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and only 2.8% either a Dor an F. General Mathematics along with 
General Science show the largest percentage of failures as defined by 
thergrades of Dor BE) (25.9% and 22.6% respectively). “This large 
percentage of low grades raises some concerns over the effectiveness 
of the curriculum and instruction in these subject areas, or of the 
assessment procedures used which may be setting unrealistic standards. 
It is also evident from the table that larger percentages received 
grades of D or F in Social Studies (20.3%) than in either of the 
English streams (16.2% and 17.1%), suggesting the need for more 
in-depth examination to determine the source of these differences. 
Ratings of subject area objectives. For the so-called external 
indicators of educational quality, namely the student ratings of the 
various educational objectives, it is evident that for most subject 
areas about 60% of the students rated the objectives as having been 
either adequately or very adequately achieved. Physical Education, 
Matriculation Mathematics, and Vocational Education had the highest 
percentages of responses in these two categories, all over 70%. Based 


on these indicators, the subject area of major concern would seem to 


be Second Languages, where the average adequacy rating by students was | 


43%, with the adequacy ratings for individual objectives varying from 
a low of 24.6% for objective #1 to a high of 72.5% for objective #3. 
Although detailed examination of the ratings for each of the 
Subjects and each of the individual objectives was not undertaken, it 
is noteworthy that for Second Languages, two of the objectives 
received relatively high average ratings while the others received 
ratings that were extremely low in relation to the other objectives. 


Further examination of these specific objectives may be of help to 
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Curriculum designers working in that area. Another subject area where 
the differences from objective to objective are quite great is the 
Matriculation Science, where three of the objectives had been rated at 
least adequately achieved by fewer than 50% of the students while the 
Overall average adequacy rating in this subject area was greater than 
65%. More in-depth evaluations would be required to determine the 
causes of these discrepancies. 

Summary 

Provincial norms computed for the general quality indicators 
showed 65% achieving a high school diploma and 35% attaining 
matriculation status. The average high school mark was 61.6%. The 
career based indicators of quality education included provincial norms 
of 1.7 \years of further education and 65% satisfied or very satisfied 
with their career. The average rating over all 12 goals of education 
was 55% adequate or very adequate. 

The subject area quality indicators computed provincially 
included measures of the extent of participation by students, levels 
of achievement in the subject areas, and ratings of the subject area 
objectives. The highest average number of credits in the subject 
areas was recorded for Matriculation English, with Business Education 
and Social Studies following. The highest percentage of students 
taking at least one course was found within Social Studies, followed 
by Physical Education. The average grade achieved varied from a high 
of 73% in Music, to a low of 59% in Matriculation Science. Adequacy 
ratings of the objectives were highest for Physical Education and 


lowest for Second Languages. 
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Sub-problem 5: to identify, for selected indicators of 
quality education, differences among groups of students 
Classified on the basis of student programs and student 
Characteristics. 

Rarlier analyses revealed that of the student-based variables, 
the student's sex, previous achievement, career plans, and school 
program were among the variables most closely associated with variance 
in the quality indicators tested. This section, along with the 
subsequent section, focus on sub-problem five. Here, the quality 
indicators under study are disaggregated to selected sub-groups based 
on the independent variables identified earlier. These 
disaggregations (computations of the quality indicators for sub-groups 
within the sample) follow; first focusing on student characteristics, 
next focusing on student programs, and finally, focusing on various 
combinations of these student-based antecedent and transactional 
variables. This section deals with the general quality indicators, 
while the disaggregation of the quality indicators specific to the 
various subject areas is reported in the subsequent section. The 


BeCVOBEENG On GseSt bass Dased OneinStacubionals fFacCkors 19 dealt with in 


Ene stacer part cof the chapter. 


General Indicators and Student Characteristics 
Table 10 presents a cross tabulation of both internal and 
external general indicators of educational quality based on selected 

Student characteristics. 


Sex, Examination of these results reveals females on the average 


scoring higher on just about every quality indicator. Over 10% more 
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of females than of males achieved a high school diploma (71% versus 
60%), and 4% more of females than of males achieved matriculation 
Status (33% versus 29%). Over the entire sample, the average high 
school mark was 5% higher for the females than the males (64% versus 
59%). For the career achievement indicators, males showed higher 
average number of years of further education (2.0 versus 1.4) anda 
higher percentage of respondents earning over $12,000 per year (46% 
versus 15%). However, unemployment was very slightly lower (5.7% 
versus 6.0%) for females and the job prestige rating was higher by 6 
1/2 points on the Pineo-Porter scale. Females in general also rated 
the goals of education more adequately achieved for them than did 
males. The overall goal rating was four percentage points higher for 
females and all but one of the individual goals (#2, concerned with 
using information) were generally rated higher by females. 

Previous achievement. The sample was divided into three 
approximately equal sub-samples based on the average of grade nine 
achievement. Approximately 21% more of students in the middle prior 
achievement group than of those in the low prior achievement group, 
and 21% more of students in the high group than of those in the middle 
group, attained a high school diploma. In relation to percent 
achieving matriculation status, less than 2% of the low prior 
achievement group did so, with 21% of the students in the middle group 
attaining matriculation standing. This percentage jumped to 71% for 
the high prior achievement group. The average high school mark also 
increased as one moves from low to high prior achievement groups 
(56.4% for low achievers, 59.9% for medium achievers, and 69.2% for 


high achievers) as shown in Table 10. 
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With the exception of the unemployment indicator, the career 
achievement indicators for the three sub-groups also increased as one 
moves from low to medium to high prior achievers, but the differences 
between groups are not as dramatic as for the previously mentioned 
internal quality indicators. The lowest unemployment rate (3.7%) was 
found for the middle group, with 5.7% and 7.6% unemployment for the 
low and high prior achievement groups respectively. The average years 
of further education ranged from .8 for the low achievement group to 
2.4 for the high group. The job status level increases from group to 
group. The percentage of persons earning over $12,000 per year is 
shown to be slightly higher for the high achievement group than for 
the low and middle groups which are about the same at approximately 
29%. The level of career satisfaction seems to have been 
approximately the same for each achievement group. 

The overall goal ratings are about the same for the low and 
middle prior achievement groups but about four percentage points lower 
Pon the high achievement group. Examination of the individual goal 
ratings however, shows that some of the goals (#1, #2 and #5) were 
rated higher by the high achievement group. Goals #4, and #6 - #12 
were rated higher by the low and middle achievement groups. Perhaps 
this difference was a result of the school program followed by the 
high achievement students. This possibility is examined later in the 
chapter. 

Career plans. The quality indices for each of the sub-groups, 
which were created on the basis of career plans of the students, are 
also shown in Table 10. The substantial differences among means for 


the four career groups in percent achieving a diploma, percent 


achieving matriculation status and high school marks attained reveals 
the strong relationship which seems to exist between student career 
plans and high school achievement. Less further education, but a 
lower unemployment rate is evident for students who had planned to 
enter work immediately after high school (work column). While a 
somewhat higher percentage of university bound students achieved 
higher levels of job status, wages, and career satisfaction they also 
showed higher unemployment rates. 

A higher percentage of students who had planned to go to college 
rated most of the goals as having been adequately or very adequately 
achieved. This generalization also applied to the overall goal 


rating. 


General Indicators and Student Programs 
The school program, or specific cluster of courses taken by each 
student, has been shown earlier as the one variable most associated 
with variance in the quality indicators examined. This association is 
most evident when the influence of other variables is controlled, as 
reported in the preceding chapter. Table 11 compares the computations 
of the general quality indicators for the four sub-samples based on 
student program. The four subsequent tables show the quality 
rai caters for each program further broken down by the student 
Characteristics reported on earlier. This type of double 
disaggregation was carried out to examine the extent to which each 


program "serves" different groups of students. 


Internal indicators. A higher percentage of students in the 
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Matriculation program achieved a diploma (84%) and matriculation 
standing (56%); in addition the high school average was highest for 
these students (65.7%). Rough distributions for the other important 
independent variables are shown at the bottom of Table 11 and must be 
considered in the interpretation of these comparisons. For example, 
52% of those in the Matriculation program were females, 55% were from 
the high achievement group, and 48% reported plans to go to universiy. 

The percentage of students achieving a diploma who had followed a 
General program was 20 points lower than for those in the Vocational 
program and 30 points lower than those in the Business program, all of 
which had students with similar ability and prior achievement levels 
as shown in Table 11. Very low percentages of students achieved 
Matriculation status from programs other than the Matriculation 
program. Even in this case, only 55% actually achieved matriculation 
status. The average high school mark was 63% for the Business, 58% 
for the Vocational, and 55% for the General programs. 

External indicators. The average number of years of further 
education was higher for Matriculation students and Vocational 
students and lower for General and Business students. Respondents 
from Business and Vocational programs reported no unemployment, while 
the rate was 5.6% for students from the Matriculation program and 
10.2% for students from the General program. Students from the 
Matriculation, Business, General and Vocational programs reported 
being employed in vocations having prestige ratings of 49.4, 44.3, 
41,1 Lara respectively. Fifty three percent of graduates of the 
Vocational program reported earning $12,000 or more annually, whereas 


just over 31% of those from each of the Matriculation and General 


programs did so; for the graduates of the Business program the figure 
was 13.74. The percentage for the career satisfaction item followed 
much the same pattern with 73% of Vocational graduates reporting that 
they were satisfied or very satisfied with their chosen careers and 
66% of Matriculation graduates, 61% of General graduates, and 58% of 
Business graduates make this claim. 

The overall goal rating was found to be highest for the Business 
program with a 64% adequacy rating, followed by the Vocational program 
at 59% and the General and Matriculation programs nearly identical at 
54%. The ratings of seven of the 12 individual goals (#1, #3, #5, #7, 
#9, #10, #12) were similar, with the adequacy rating for the Business 
program highest, followed by the Vocational program, followed by 
nearly identical ratings in the Matriculation and General programs. 
Exceptions to this pattern were as follows: 

a. goal #2 (use information) - the Matriculation rating was 

second to that of Business, 

b. goal #4 (health, fitness, safety) - the ratings for all four 
programs were very close, 

c. goal #6 (desire learning) - the rating for the Matriculation 
program was highest and the rating for the General program was 
notably lower, 

d. goal #8 (good citizenship) - the General rating was second to 
EnAtwNOnebUSINe Ss 

e. goal #11 (management skills) - the General rating was second 
to that of Business. 

Goal ratings within each program which stand out in that they differ 


Significantly from the ratings established by the overall provincial 
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norms, included: 

a. Business - goal #7 (understand changes) and goal #11 
(management skills) were rated higher in relation to the other 
goals than would be expected based on the norms, 

b. Vocational - goal #7 (understand changes) was rated higher in 
relation to the other goals than would be expected based on 
the norms, 

c. Matriculation - goal #6 (desire learning) was rated higher in 
relation to the other goals than would be expected based on 
the norms, and 

ad. General - goal #8 (good citizenship) and goal #11 (management 
skills)were rated higher, while goal #6 (desire learning) was 
rated lower in relation to the other goals than would be 


expected based on the norms. 


Student Programs and Student Characteristics 

A further breakdown of the above data according to the various 
student characteristics is presented for each of the four student 
programs (Business, Vocational, Matriculation, and General) in Tables 
12 through 15. Following are attempts to point out only major areas 
of difference or Similarity. Where the numbers of students were 
considered too small for meaningful comparisons the results were 
omitted from the tables. 

Internal indicators: sex of respondent. In relation to the 


percentage achieving a high school diploma, females were higher in all 


four programs. However, the difference between the sexes was greatest 
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in the Business program 1%) and in the Vocational program (9%). The 
ditference between male and females was only 5% in the Matriculation 
and General programs. 

In relation to the percentage achieving matriculation Status, the 
difference between the sexes was virtually non-existent for those in 
the Matriculation program (Table 14). In the cases of both the 
Vocational and Business programs, a higher percentage of males 
(approximately 2.5%) than of females achieved matriculation status. 
The reverse was found for students enrolled in the General program. 

The average marks achieved were three to four percentage points 
higher for the females, irrespective of program. 

Internal indicators: previous achievement. There was almost no 
relationship between prior achievement and the percentage achieving a 
high school diploma for students in either the Vocational or Business 
programs. The higher percentage shown in the case of the high prior 
achievement group within the eee program may have its source in 
the small numbers involved. The relationship was much more evident in 
the cases of both the General and Matriculation programs where the 
percentage increases approximately 10 points from the low to medium 
and from the medium to high prior achievement groups. 

Since the overall percentage of Business students attaining 
matriculation status was small, further breakdown on the basis of 
prior achievement was deemed to be of questionable value. In the 
cases of the remaining three programs (Vocational, Matriculation, and 
General) the increase in the percentage of students achieving 
matriculation status was very large as one moves from the low, to the 


medium, to the high achievement groups. The increase found was much 
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larger in these cases than for the above-mentioned indicator of the 
percentage achieving a high ote diploma. 

The average marks achieved showed a consistent relationship with 
the prior achievement variable over all four programs. The increase 
was greater between the medium and high achievement groups than 
between the low and medium groups for all four programs. 

Internal indicators: career plans. The relationship between the 
percentage achieving a high school diploma and the career plans of the 
Student displays considerable similarity over the four programs. 
However, two important differences between programs may be noted in 
the comparison of the results presented in Tables 12 through 15. For 
the General and Matriculation programs, a higher percentage of 
students with "other" career plans than students with "work" as a 
career plan, achieved a high school diploma; the converse was found in 
the Business program, and in the Vocational program there was no 
difference between the two career plan categories. No difference in 
the percentage achieving a high school diploma was found between the 
university and college bound student taking a Vocational progran, 
while higher percentages of university bound students achieved a 
diploma in both the General and Matriculation programs. 

Comparisons between the four career plan groups yield similar 
patterns in the case of the percentage of students achieving 
matriculation status. 

Similarily, in relation to the average marks achieved, the 
relationship with career plans was generally the same from program to 


program. 
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External indicators: sex of the respondent. In relation to the 


various career achievement and goal rating variables, Tables 12 


through 15 display the following important differences among the four 


programs when examining the variable of sex. 


ic 


The percentage unemployed was reported higher for female than for 
males from the General program (12% versus 9%), while slightly 
higher unemployment was reported for males (a difference of .5%) 
from the Matriculation program. 

In the case of the indicator "career satisfaction", the 
percentage of females reporting either satisfied or very 
satisfied was much higher than for males (71% versus 53%) among 
the General program graduates. This indicator shows the reverse 
for graduates from the Matriculation program with males slightly 
higher than females (67% versus 65%). 

The differences between males and females on the ratings of the 
adequacy of the various goals were similar for the four programs. 


External indicators: prior achievement. In examining the pattern 


of achievements (as represented by the various external indicators) by 


the low, medium, and high prior achievement groups within each 


program, the following comparisons stand out. 


thee 


In the case of the Business graduate, prior achievement appears 
to be not related to the years of further education, as is the 
case with the graduates from the other three programs. 

The percentage unemployed is highest for the high achievement 
group and lowest for the medium achievement group within both the 
Matriculation and General programs. Comparisons with the 


Business and Vocational programs are not possible due to the zero 
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unemployment reported from graduates of these programs. 

The percentage earning over $12,000 per year indicates a positive 
relationship with prior achievement in the case of the Business 
and Vocational programs, but a negative relationship in the 
General program. From the Matriculation program, the low 
achievement group shows the highest percentage earning over 
5423000. 

In relation to career satisfaction, the high prior achiever from 
the General program reports much lower career satisfaction on the 
average than was found for graduates of the other three programs. 
In comparing goal ratings for the three prior achievement groups, 
Similarities exist within each of the four programs in that the 
medium group rates the overall goal achievement higher in the 
Matriculation, Business, and Vocational programs whereas the low 
achievement group rates the goals highest in the General program. 


External indicators: career plans. The relationship between 


Career plans and the external indicators appears to differ 


Significantly from program to program in the following instances. 


4 


Within the General program, the percentage reporting the status 
of unemployed and seeking a job was highest for those who had 
plans to attend college and lowest with "other" career plans. 
Within the Matriculation program, the reported unemployment rate 
was highest for those with "other" career plans and lowest for 
those who had planned to attend university. 

Within the General program, the percentage earning over $12,000 
was highest for students with "other" career plans and lowest for 


those with plans to attend university, whereas within the 
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Matriculation program the highest was for those who had planned 
to attend university. 

Si In relation to career satisfaction, the highest measure was found 
for those who had planned either college or university within the 
Matriculation program, whereas within the General program the 
highest measure was found for those who did not plan either 
college or university. 

4. Overall goal ratings were highest for students in the Business 
program who had planned to go to work, in the Vocational program 
for those who had planned to attend college, and lowest for those 
in the General program who had planned on going directly to work. 

Summary 
The disaggregation of selected general quality indicators of 

quality education based on the student variables most closely 

associated with variance in these indicators has been presented in 
this chapter. The findings presented revealed the differences between 
male and female achievements with females scoring higher on most 
indicators. The extent of the difference among the three groups of 
students categorized on prior achievement were also identified, with 
differences of up to 20 percentage points found in favor of the high 
prior achievement group. The reported unemployment rate was an 
exception, with the high prior achievement group showing the highest 
unemployment rate. The nature of the association of the student 
career plans with the various indicators of quality education was also 
portrayed. 

Disaggregations based on student program revealed notably higher 


percentages of students in the Business and Vocational programs 
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achieving a high school diploma than was the case for the General 
program although other variables, such as prior achievement, were very 
Similar for the three programs. Average high school marks were also 
higher for the Business and Vocational programs compared to the 
General program. Unemployment rates were reported as zero for both 
the Business and Vocational programs and graduates of the Vocational 
program had the highest percentage reporting a wage of over $12,000 
per year. 

The slevels reported on “the iquality indicators for groups 
Categorized by student characteristics within each of the student 
programs revealed further differences. The male/female differences 
were found to be more pronounced in some areas and even reversed in 
others, depending upon the program that the student was enrolled in. 
Prior achievement was not found to be a factor in the completion rates 
for a high school diploma in either the Vocational or Business 
programs. The average marks however showed a consistent relationship 


over the four programs. 


Subject Area Quality Indicators 
Disaggregated According to Student Characteristics 
This section continues the report of the findings in reference to 
sub-problem five, focusing on the subject area quality indicators and 
the differences for various groups of students based on student 
Characteristics. 


Quality indicators selected with reference to subject areas 


included two types of internal indicators and the external rating of 


subject area goals by students after they had left school. Due to the 
low numbers, further disaggregation of the external indicators by the 
various student characteristics was not undertaken. The internal 
indicators, measures of the extent to which each Subject area was 
providing instruction to students and measures of the level of 
achievement by students within the subject area, were disaggregated 
based on the student characteristics found associated with variance aial 
the indicators. The results of these disaggregations follow, and 
present some indication of the way that each subject area has provided 


for students of varying characteristics. 


English Subject Area 

The calculations of the selected indicators for the various 
sub-groups of students in both the Matriculation English and General 
English subject areas are presented in Table 16. 

Sex of student. Slightly higher percentages of females than of 
males took at least one Matriculation English course, and of those 
that did, females averaged slightly more credits than did males. The 
converse was the case for the General English subject area. Females, 
on the average, achieved marks 64 to 7% higher than males in both 
General and Matriculation English. The percentage of females 
achieving grades of A or B was 6 points higher than for males in the 
Matriculation route and 7 points higher in the General route. An 
examination of those receiving grades of D or F shows an even wider 
spread (12% in the Matriculation and 15% in the General). Marks also 


show a wide advantage to females, more so in the General than in the 
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Matriculation program. 

Prior achievement. When prior achievement is considered, a 
positive relationship is evident with the number of Matriculation 
English credits taken. The reverse was true concerning the 
relationship between prior achievement and the number of credits in 
General English. For the Matriculation English route, 38% of the low 
prior achievement group and 20% of the middle achivement group were 
assigned D and F grades, whereas only 5% of the high achievement group 
attained these low grades. The low and middle prior achievement 
groups fared more favorably in General English with respective figures 
shown being 22%, 12%. 

Career plans. When the indicators are examined for the 
sub-groupings based on student career plans, the general trends 
evident in Table 16 show proportionate decreases in the percentage of 
students enrolled, and the achievement levels of those enrolled, as 
one moves from university to college, to work bound students. In 
General English, the reverse trend was found to be the case. Those in 
the "other" category show levels comparable to the college group in 
both General and Matriculation routes. 

The General English program appears not only to attract the 
enrollments from the lower achievement groups and the work bound 
student, but also evidence is given that these types of students 
achieve better in this program than in the Matriculationsprogram.= To 
some degree at least the two programs complement each other in 


providing English instruction to the whole student body. 
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Table 17 presents the internal indicators for the Social Studies 
and Social Science program, disaggregated by the student variables 
selected. 

Sex of student. Little difference is evident between the sexes 
on the extent of participation in these courses. Achievement 
indicators however show females receiving higher grades than males in 
both subject areas, with the average mark 3% higher in Social Studies 
and 7% higher in Social Science. The differences found for the 
percentage achieving either an A or B was also considerable, 
especially in Social Sciences where 28% of the males compared with 53% 
of the females received these high grades. The differences between 
males and females were also greater in the Social Science subject 
area, when comparing the percentage of students achieving grades of 
either a) or F. 

Prior achievement. Students from all three levels of previous 
achievement tend to take at least one Social Studies course, while the 
high achievement group take more credits than the lower groups. The 
Social Science courses were taken more often by the lower prior 
achievers, with 69% of the low achievement group taking at least one 
course, and each of these taking 1.8 such courses on the average. 
Similar to the English subject area, students from the lower previous 
achievement groups do achieve better in Social Science, where more of 
them are enrolled, although higher percentages (6% more) of the low 
group received grades of D or F in these "General" courses than in the 


corresponding General English. 
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Career plans. Once again, when focusing on student career plans, 
the results in the Social Studies and Social Science subject areas are 
Similar to those shown in English, except that the Social Studies area 
serves to a limited extent all students irrespective of their career 
plans. In Social Science however, 75% of work bound students enroll 
in one or more courses, in comparison with only 40% of the university 
bound students. Achievement levels are also lowest for the work bound 


Student in both areas. 


Mathematics Subject Area 

Sex of student. Both Mathematics routes appear to provide 
instruction to equal numbers of males and females, as shown in Table 
18. Also, while the achievement indicators for females are higher 
than for males, the differences are less than those noted earlier for 
English or Social Studies. 

Prior achievement. Twenty-five percent of the low achievement 
group, 774 of the middle group and 98% of the high achievement group 
took one or more Matriculation Mathematics courses. A Similar 
relationship in the reverse direction was found for the General 
Mathematics courses. Higher percentages of the lower achievement 
groups received D's and F's in both programs than in any of the other 
subject areas. Based on these figures, the General Mathematics 
program particularly would appear to be not meeting the needs of the 
poorer student as well as the English and Social programs. 

Career plans. The Matriculation program serves 96% of the 


university bound student group and only 36% of the students planning 
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SUEIOIMNG eT rectly |Comwock Only 18% of those with university plans 
and 80% of those planning on entering work directly after high school 
took General Mathematics courses. Over 25% of those work bound 
received failing grades. The college bound student and those in the 
"other'' category fell approximately half-way between the university 


and work groups on all indicators. 


de, ~ 


Science Subject Area 

sex of Student... The Matriculation Science subject area is the 
only Matriculation program enrolling higher percentages of males than 
females, aS shown in Table 19. Both sexes were about equally 
represented in the General Science courses. Once again the females 
received higher marks, especially so in the General program where the 
difference between average marks was 5%; also 15% more females 
received A's and B's, and 10% more males received grades of D and F. 

Prior achievement. Fifty percent of the low prior achievement 
group and very high percentages of the remainder of the students took 
at least one course from the Matriculation Science subject area, with 
the high achievement group averaging 22 credits per student, the 
highest of all subject areas. This can be compared with averages of 
17 and 14.5 credits for the Matriculation English and Mathematics 
subject areas respectively. The General Science set of courses 
provided instruction for 78% of the low achievement group but a 
smaller percentage of the middle and high achievement groups than, for 
example, the General Mathematics courses. Previous achievement again 


is associated with marks in Science, with 43% of the low achievement 
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group receiving grades of D or F in the Matriculation route and 25% 
doing so in the General route. Nearly the same percentages of the 
high previous achievement group received A's and B's (68%) in 
Matriculation Science as in the other matriculation subject areas. 
Career plans. Very similar patterns to that of Mathematics for 
the involvement and achievement indicators are evident in the various 
groups identified by career plans. That is, the Matriculation courses 
provided instruction to very high percentages of students with plans 
of going to university, but also to approximately 35% of those 
Planning on going directly into work after high school. The General 
Science courses provided instuction to 70% of those planning on work 


and 35% of those planning collegé after high school. 


Second Languages Subject Area 

The Second Languages subject area combines several different 
language programs having the same subject area objectives. 

Sex of student. Table 20 shows 12% more of the females than of 
males registered in these courses, with an average mark nearly 7% 
higher. Ten percent more males received D's and F's than females and 
16% more females than males received marks of A or B. 

Prior achievement. Once again higher percentages of the higher 
achievement groups were enrolled in these courses and the higher 
achievement groups had taken more courses, and completed more credits. 
The average mark for the high achievement group was 10% higher than 
for both the middie and low groups whose average achievement was about 


equal. However, 27% of the low achievement group received grades of D 
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or F, in comparison to 20% for the medium group and 8.5% for the high 


group. 


Fine Arts Subject Areas 

wex Of (student = "The figures injTeble 2iereveal almost equal 
enrollments by males and females in the Art courses, whereas Drama and 
Music had higher percentages of females. For all three of these 
subject areas, the average number of courses taken by those of both 
sexes who were enrolled was about the same. The males in Art, 
however, received lower grades than their male counterparts in Drama 
and Music, with 20%, 12%, and 4% of the males receiving grades of D or 
F respectively. The average Music mark was high as reported earlier 
in this chapter, with 80% of the males and 90% of the females 
receiving a mark of either A or B. The difference between males and 
females on the achievement indicators was greatest in Art, where 44% 
of the males, compared with 65% of the females, received grades of A 
or B; and 20% of males, compared with 8% of females, received grades 
Go DPorir. 

Prior achievement. Nearly equal percentages of low and medium 
achievement students enrolled in Art courses with slightly lower 
percentages from the high achievement group. Differences in Drama 
were less pronounced. ‘The distribution of numbers enrolled in the 
three subject areas reveals higher percentages for the high prior 
achievement group in Music than in either Art or Drama. In all three 
Subject areas and for all three prior achievement groups the average 


mark increases aS one moves from Art, to Drama, to Music. Overall, 
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13.8% of the Art students received D's or F's, while comparative 
figures for Drama and Music were 7.9 and 2.8 respectively. 

Career plans. Both Art and Drama provided instruction to higher 
percentages of students with college, work, and "other" post high 
school plans than to students planning to go to university. fhe 
reverse was true for the Music program. The achievement indicators 
(average marks and grade distributions) however follow the same 
pattern as for the subject areas discussed previously, with the 
university bound student highest, the student planning work lowest, 


and the college and other student in the middle. 


Physical Education Subject Area 

Sex of student. When the results presented in Table 22 are 
compared with those for the other courses mentioned thus far, it is 
evident that Physical Education is the only subject area having both 
approximately equal participation levels and approximately equal 
achievement levels for males and females. 

Prior achievement and Career plans. Participation levels are 
also almost equal for all three prior achievement groups and for all 
four categories of students, based on career plans. Achievement 
patterns evident for most of the other subject areas are repeated 
here, with the high prior achievers having the highest average marks, 
and more of the university bound students (70%) achieving A's and B's 
whereas a smaller percentage of the students who planned to enter work 


(51%) achieved these grades. ~For D's and F's the percentage was 


lowest for the university bound and highest for those aspiring to work 


ee. 
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on leaving high school. 


Business Education Subject Area 
"” sex of student. As evident from a perusal of Table 23, 92% of 
all females took at least one course in Business Education, whereas 
the comparable figure for the males was 77%. The females also 
averaged nearly 3.9 courses in the subject area for an average of 16 
credits, while the corresponding figures for the males were 2.2 
courses and almost 6.9 credits. The average mark received by females 
was six percent higher than that for males with 15% more females than 
males receiving grades of A or B. Twice as many males as females were 
given grades of D or F. 

Prior achievement. ~The percentage of students taking one or more 
courses was over 80% for all three levels of previous achievement; 
however the low achievement group enrolled in more courses and 
achieved more credits than the high achievement group. Achievement 
levels in Business Education increased progressively from an average 
mark of 57% for the low prior achievement students, to 63% for the 
middle group, and 70% for the high group. 

Career plans. Over 80% of students in each of the four career 
plan alternatives had enrolled in one or more courses. Students who 
planned to enter the work force directly following high school 
completed more courses and earned more high school credits (18 on the 
average), whereas college bound and "other" students each earned about 
12 credits and the university bound students less than 7 credits on 


the average. Students who had planned to go to university received 
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higher average marks than did those in the other three career 
aspiration groups. This last finding was similar to that for all 


other subject areas. 


pex Of ‘Student. Figures an Table 24 reveal that 50% of the 
females and 6% of the males were enrolled in the Home Economics 
subject area, while for Industrial Arts these percentages were 
reversed. Thirty-five percent of the males and 13% of females had 
taken at least one Vocational Education course. Achievement levels 
for each of the three areas again were highest for females. In Home 
Economics, 25% of the males received grades of D or F compared with 8% 
of the females. In Vocational Education, 76% of the females received 
A's or B's, compared with 48% of the males. 

Prior achievement. All three programs had about 1/3 of the low 
achievement group and about 1/4 of the middle group taking at least 
one course. Twenty percent of the high achievement group were 
involved in Home Economics and Industrial Arts but fewer in the case 
of Vocational Education. The differences on average marks and grade 
distribution was less for Vocational Education than for either 
Industrial Arts or Home Economics when comparing one prior achievement 
group with the next. 

Career plans. Equal percentages of those who planned to go to 
work immediately following high school were involved in each program 
although those in Vocational Education took more courses on the 


average. Differences in achievement levels among the groups based on 
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career plans was shown to be greatest in the case of Home Economics 
where 81% of the university bound students achieved either A or B. 
Average marks across the four groups were quite Similar for both 
Industrial Arts and Vocational Education. 

Summary 

Quality indicators dealing with the extent of participation of 
students and the level of achievement by students within each subject 
area have been presented, comparing provincial norms with figures for 
groups of students categorized according to the selected student 
Characteristics. 

The extent to which the sex of the student was associated with 
achievement levels was found to be less in some subject areas than in 
others. Differences were also identified in the level of 
participation of students from the three prior achievement groups in 
the various subject areas and the extent to which each group reached 
various levels of achievement within each subject area. The different 
levels of participation and achievement were also identified for 


groupings of students based on their career plans. 


Sub-problem 6: to develop profiles for selected schools 
and, districts using tne various indicators of quality 
education. 
The previous section has demonstrated in more detail the nature 
of the influence of the student based independent variables which had 
been demonstrated in Chapter 5 to be associated with variance in the 


quality indicators. Such information could be used to direct more 


in-depth evaluations which may determine reasons for the different 
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achievement levels by different student pcedes and thereby identify 
areas where changes may be deemed desirable. 

In this section the focus is shifted to regional, system and 
subsystem factors. Analysis of variance reported in Chapter 5 showed 
important associations between the variance in the quality indicators 
and school districts and to a greater extent schools. These variables 
were not included Tr chem Mules pike Classification Analyses due to the 
internal limitations of that procedure. Further, since school 
districts and schools represent the two major decision making levels 
in implementing provincial policies and curriculum, it was considered 
desirable that in a province-wide monitoring activity they should also 
be recipients of evaluative information specific to their sphere of 
influence. It was also considered important that where possible the 
information available would show the educational achievements for the 
programs offered relative to important student characteristics 
identified earlier. 

The approach chosen for illustrative purposes was to select nine 
school districts and twelve schools as examples and to develop 
profiles for each on selected general and subject area indicators. 

The districts and schools are identified in the tables and text only 
by number, but were selected so as to represent a variety of systems 
and schools as well as each region within the province. The 
discussion that is presented with these profiles attempts to 
demonstrate the differences that exist from one system or subsystem to 
another, and to illustrate the type of information that could be 
available to schools and school districts to assist them in achieving 


an awareness of differences that exist between jurisdictions and to 
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allow comparisons with provincial norms. Analysis of variance had 
SHOWER EELe Oe NO predictability Of Andrtcators nae the geographic 
region aS a factor. Nevertheless, from a provincial monitoring 
perspective, profiles were also developed and presented as a check on 
possible regional desparities. 

The discussion related to each set of the illustrative profiles 
for each quality indicator focuses on jurisdictions with the lowest 
and highest scores on the indicator and examines notable differences 
in patterns when comparisons are made with the provincial norms 


reported earlier. 


Percent Achieving a High School Diploma 

Provincial norms established earlier show that 65% of the 
students achieved a high school diploma. Disaggregations of the 
entire sample found a difference of 10 percentage points between males 
and females, 20 percentage points between each of the low, medium and 
high prior achievement groups, and 30 percentage points between 
students planning to work immediately following high school, and those 
planning on attending university. 

Regional profiles shown in Table 25 reveal a low of 63.5% in 
Region #2 and a high of 69% in Region #3 of students achieving a 
diploma. The differences between males and females in the completion 
rates were lower than the norm for Region #1 and higher than the norm 
for Regions #5 and #6, the northern regions of the province. Region 


#5 shows achievements substantially higher than the provincial norms 


for college and university bound students. Achievement rates were 
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5%-10% lower for college and work bound students in Region #2, while 
considerably higher percentages of students planning on work following 
high school in Region #3 achieved a high school diploma. 

School Districts. Percentages of students achieving a diploma 
ranged from 38% to 78% in the nine school districts selected for 
illustrative purposes, as shown in Table 25. The difference between 
males and females was 25% for District #1, while for District #8 the 
completion rates for males and females were the same. 

District #2 had the lowest percentage of students achieving a 
high school diploma at 38%. Notably poorer completion rates were 
recorded for both the lower and higher prior achievement groups. 
Completion rates for students planning to work (0%) and students 
planning to attend college (50%) contrast with those who reported 
plans for university or "other", where the percentages achieving a 
high school diploma were much higher than the provincial norms. 

The highest achievement of diplomas was recorded for District #4, 
at 78%, primarily with higher Bere arenne by females (90%). Students 
Of al prior vachievemen= groups did better win) this district, but 
particularly the medium group. For the career plan groups, the 
college bound students were the only group below the norm, with 
particularly high achievement levels for students planning direct 
entry into work, and those bound for university. 

The high prior achievement group of students, and the university 
bound groups appear to be well short of provincial norms in District 
#3, with 20% fewer students in the university bound group completing a 
Giploma. Students with "other'' career plans were also well below 


norms, while college and work bound students show diploma completion 
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rates higher than the norm. 

One large school district (#6), had notably low percentages of 
the low prior achievement group and of the work bound students 
achieving a diploma. District #8 also had a low percentage of 
students completing the diploma who were among those planning to work 
immediately following high school. 

Schools. Diploma completions within the illustrative schools 
ranged from 31% up to over 78%. Male and female completion rates 
varied considerably from school to school, ranging from School #3 
which had 35% more females than males completing diploma requirements 
to School #12 with 19% more males than females doing so. 

Two schools with very low percentages of students achieving a 
diploma were schools #7 and #11. In School #7 both male and female 
percentages were low and all three prior achievement groups were 
markedly below the respective norms. Students from CU DOUG Ca be Sis 
plan groups were also below their norms, with those planning to enter 
college or work following high school being furthest below the norm. 
In School #11, females particularly dropped below provincial norms. 
Students from the middle prior achievement group, and those planning 
on attending college had completion rates above average. No students 
who were planning on entering work directly after high school 
completed the high school diploma in the sample from School #11. 

Over 78% of students in two of the schools completed diploma 
requirements. All categories of students in School #2 had completion 
rates above the norms with the exception of the group of students 
planning entry into college. The low prior achievement group had low 


complietion rates at School #6, as did students whose career plans were 


college or "other". 

School #12 displays an atypical profile when compared with the 
others in that completion rates for males were 15% above average and 
rates for females were 13% below average. This school also had the 
lowest completion rate for the low prior achievement group and poor 


results for all students not planning on attending university. 


The average of all high school marks for all students in the 
province was 61.6%. Females received marks close to 5% higher on the 
average than did the males. Marks were 5% higher in the medium prior 
achievement group than in the low, and 10% higher in the high group 
compared to the medium. Students with plans to attend university 
received marks higher than did those in the three remaining career 
groups. 

Regions. Examination of Table 26 shows no important differences 
in the overall means of high school marks among the six geographical 
regions. Also, no substantial differences from region to region are 
evident for the analyses by sex of oar pa by prior achievement 
level and career plans. 

Districts. Differences among the school districts were small 
with the possible exception of District #4, which stood out from the 
others in that the average high school mark was 4% above the 
provincial norm. For this district, students of both sexes and all 
three prior achievement groups, as well as most groups based on career 


plans, received marks higher than provincial norms. As reported in 
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the previous section, District #4 was one of the two districts having 
the highest percentage of students completing a high school diploma. 

Schools. The two schools with the highest average high school 
mark (#2 and #6) were the ones shown in Table 25 as having the highest 
percentage of students achieving a diploma. Schools #7 and #11 had 
the lowest average grades and were the two schools which had the 
lowest completion rates for the high school diploma. 

For School #2 the mean of the marks for females was the highest 
of the illustrative schools selected and that for males only slightly 
above the norm for males. Average marks were above provincial norms 
for the low achievement group but slightly below for the other two 
prior achievement groups; above the norm for students with career 
plans other than college and slightly below for the college bound 
group. For School #6, average marks for all three prior achievement 
groups were below the provincial means yet the overall mean for the 
school was second highest among schools and above the provincial norm. 
This could only be explained by the school having unusually high 
proportions of their students within the high prior achievement group. 
Students with university plans at this school appear also to have 
achieved higher average grades when compared with the provincial 
average. 

For School #7, students from all groups within both the prior 
achievement and career plans categories had average marks well below 
provincial averages. For School #11, on the other hand, marks for the 
medium and high prior achievement students were close to provincial 
means but for the low achievement group they were about 3% below. For 


students who had planned to attend college after leaving high school 
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marks were above average, whereas for the other three career groups 
they were below the provincial averages. 

Two schopis (#8 and #10) nad male students with higher average 
marks than those of the female students. ThiS was atypical and may 
have been related to the accompanying higher than average marks for 


students planning to enter work. 


Disaggregation of the quality indicator "Percent Achieving A High 


School Diploma" was undertaken for regions and for representative 


school districts, based on the ability of the student and the type of 


program followed in high school. In order to assist in interpreting 
the findings for the various sub-groups, corresponding enrollment 

figures were also provided. These figures are presented in Table 27 
and reflect enrollment patterns within regions and districts for the 
Business, Vocational, Matriculation and General programs broken down 


further by the ability level of the students. The percentages of 


students achieving a high school diploma within each sub-group, may be 


indicative of extent to which each program was providing an 

siege aoe educational experience for the particular ability group 
within each region and/or district. The numbers displayed prior to 
the slash (/) are percentages of that ability group enrolled in the 
particular program. The numbers following the slash (/) are the 
percentages of students within that specific group achieving a high 


School diplomas 


While figures are presented for the six geographical regions and 
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the nine school districts, for purposes of example, attention is 
directed to districts #4, #6, and #7. Two of these districts (#6 and 
#7) were selected for closer examination mainly because of the larger 
N's and the third one (#4) mainly because the results differed 
somewhat from those of the other districts. Also evident in the table 
is the absence of Business and/or Vocational programs in some 
dqdistricts: 

District #4. The figures displayed for District #4 show notably 
higher percentages of students enrolled in the Business (13% versus 
9%) and Vocational (13% versus 8%) programs and lower percentages in 
the General program (18% versus 31%) but about the same percentages in 
the Matriculation program, when compared with provincial norms. 
Focusing on the Business program, enrollments come entirely from the 
low and medium ability groups with percentages achieving a diploma 
much lower within the low ability group when compared with the 
provincial norm (33% versus 62%). The Vocational program within this 
Gistrict shows both a higher enrollment rate (13% versus 8%) and much 
higher diploma achievement rate (71% versus 55%), particularly for the 
low ability student where 56% were enrolled (versus 13%) and 80% of 
these achieved a diploma (versus 55%). In the Matriculation program, 
enrollments were slightly higher overall (55% versus 52%) but there 
Was zero enrollment from the low prior ability group, whereas the 
provincial average was 20% for this group. Ninety-six percent of the 
Matriculation students in this district achieved a diploma compared 
with 84% provincially. This district enrolled almost half the normal 
percentages (18% versus 31%) in the General program with a somewhat 


higher rate of diploma completion as well. 
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This profile appears to indicate that this district has a 
combination of enrollment policies and program offerings which are 
meeting the needs of students to a greater extent than the other 
districts shown in the table. One exception to this generalization is 
the Business program, particularly with respect to the low prior 
abDeELeEyY group. 

Districts #6 and #7. Comparing the two largest districts with 
each other yields some interesting findings which may be related to 
district policies and could prompt more in-depth evaluations. 

Enrollment rates show approximately equal total percentages 
enrolled in the Business program, but with District #7 enrolling 
notably more in the Vocational and General programs and District #6 
enrolling more in the Matriculation program. While total enrollment 
rates are similar in the Business program, District #7 enrolled 17% 
(compared with 14% in District #6) of the low prior achievement 
student. In the Vocational program, as in Business, District #7 
enrolled higher percentages of students from the low and medium prior 
achievement groups. District #6, on the other hand, enrolled higher 
percentages of students from all three ability levels in the 
Matriculation program. Patterns similar to both the Business and 
Vocational programs are evident in the General program, with District 
##7 enrollments higher from the low and medium achievement groups. 

Overall percentages of students achieving a high school diploma 
within the two districts had previously been found (Table 25) to be 
very close (64.1% versus 65%). The breakdowns provided in Table 27 
however, show considerable differences by program. Percentages 


achieving a high school diploma are 9% to 26% higher for District #7 


“a 7 
| ry 
: ” ‘ eye) FO ebsen AAS “uki fs 


s Annes die wee tom eco 2 tae vonage cm. ee 


+ 


abe ) . ‘oh] O19 At eefave aaa 
¢ aD bo | i= Ke bes 


= ri) (ere qt ate 


ae 


by =e ) ; J 

‘ a Pe ee 
1h ms be 
1 = 
4 
‘te e 
“ 4 ‘ 
= 
-~ 9 
Mi pt r4 A = ys 
3s ai): SE 
~ - 

i ta (241beo- 2s Gi 24 
; By - cacy wie sf 
2 . ’ j ‘ + B Ty < } af 

i s ey ret 

- ~ : ul 
- \ ‘ 4 #2? (R335 iz 
i ‘ 
tv ati Vay 
al + 


NGOLVY: S28 vane aa str aos 


- giuerny” are eo web oly meer eit is Abia OF 
—@ j " is 


z i 


; ? 


RAE 8 FA GMs oe pute Jo aegatenetiag hae 


ber ai te. al i ah Ri 


150 


in the Business, Vocational, and General programs, whereas District #6 
completion rates are higher by three points in the Matriculation 
program. 

Comparisons between Districts #6 and #7 also show considerable 
adifferences within the various groupings based on prior achievement. 
Higher percentages of the low prior achievement student, in District 
#7 achieved a high school diploma within all four programs. This 
difference was greatest in the Vocational program, where 68% of the 
low prior achievement group achieved a diploma in District #7, 
compared with 38% in District #6. For the medium prior acheivement 
groups, higher percentages achieved a diploma in District #7 in all 
programs except the Matriculation program. Students in the high prior 
achievement group showed higher completion rates in District #6 
however in all programs except the Vocational program. 

Summary 

Two selected indicators of quality education were disaggregated 
to the six regions of the province along with a few school districts 
and a few schools that were selected for illustrative purposes. The 
percentage of students achieving a high school diploma and the average 
high school mark were selected as the indicators of quality education 
Loe hperused. 

Regional differences were not great, but the percentage achieving 
a diploma was found to vary from 38% to 78% among the school districts 
and from 31% to 78% among the illustrative schools. Average high 
school marks were also shown to vary notably among the districts and 
schools. The findings also revealed differences among the schools and 


districts, in the percentage of students achieving a diploma, for 


sub-groups based on the sex of the student as well as oe jo}qnke}a 
achievement groupings. 

A further disaggregation was undertaken to show both the 
enrollments and percentage achieving a high school diploma, for the 
three prior achievement groups within each of the four student 
programs, for each of the six regions and nine districts. These 
findings revealed differences in enrollments within the various 
programs, with levels of diploa achievement being higher in some 
districts for some prior achievement groups and low for others, when 


comparing districts. 


Summary 
This chapter has presented the results of the information phase, 
which encompassed sub-problems four, five, and six. Following isa 
brief summary of the findings associated with each sub-problem. 
Sub-problem 4 


Sub-problem 4: to compute provincial norms for each 
indicator of quality education. 


Provincial norms were computed for each indicator of’ quality 
education. The general quality indicators showed 65% of the student 
group achieving a high school diploma and 30.5% attaining 
matriculation status. The average mark achieved by students over all 
high school subjects was 61.6%. 

The external quality indicators exhibited provincial norms 
including: 

a. 1.7 average years of further education, 


b. 5.8% unemployed, 
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ad. 64.4% satisfied or very satisfied with their career, ae 

e. 55.3% rating the goals overall as adequately or very 

adequately achieved, with the highest goal receiving a rating 
of 69.2% and the lowest 40.6%. 

The subject area quality indicators computed provincially 
included measures of the extent of participation by students, levels 
of achievement in the subject areas, and ratings of the subject area 
objectives. The extent of participation within each subject area 
ranged from .8 to 13.1 average credits per student and 11.3% to 99.1% 
of the students taking one or more courses. The level of achievement 
indicators displayed average grades in the range of 58.8% to 73.3%, 
with 84.7% of the students in one subject area receiving an A or B and 
32.44 of students receiving such grades in the subject area at the 
other end of the continuum. The average adequacy rating on the 
Subject area objectives ranged from 42.9% adequate or very adequate to 
76.64. 

Sub-problem 5 
Sub-problem 5: to identify, for selected indicators of 
quality education, differences among groups of students 

Classified on the basis of student programs and student 

Characteristics. 

Each of the general indicators of quality education were 
disaggregated based on student characteristics of sex, previous 
achievement, career plans, and on student program. Further 
disaggregations of the indicators based on the selected student 
characteristics were computed for each of the four student programs 


respectively. 
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The findings presented revealed notable differences between male 
and female achievements, with females scoring higher on most of the 
general indicators. Previous achievement (categorized into low, 
medium, and high) displayed differences favoring the next higher 
category of up to 20 percentage points on some indicators. One 
exception to this trend was the unemployment rate, with the highest 
rate reported for the high achievement group. 

The statistics resulting from the disaggregations by student 
program, revealed generally higher levels for the Matriculation 
program on the internal indicators but higher levels for the 
Vocational and Business programs on some of the external indicators. 
For the most part, the General program showed lower lévels on both 
internal and external indicators when compared with the other 
programs. The Vocational and Business programs had enrollments 
primarily from the low and medium prior achievement groups and 
achievement rates for these groups were notably higher than for the 
Similar student in the General program. 

Quality indicators disaggregated by student characteristics 
Within each subject area were computed, providing an overview of 
student participation and achievement by students of different sex, 
prior achievement, and career plans. Enrollment patterns revealed a 
higher percentage of females enrolled in the Matriculation routes, and 
in Second Languages, Business Education, Drama, Music, and Home 
Economics. Male enrollments were higher than females in the General 
routes and in Industrial Education. Achievement levels of females 
were generally higher than for males, but especially in the General 


routes and more so in English and Social Studies than in Mathematics. 
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Sub-problem 6 
Sub-problem 5: to identify, for selected indicators of 
quality education, differences among groups of students 
classified on the basis of student programs and student 
characteristics. 

Focusing on two of the general indicators (percentage of students 
achieving a high school diploma and the average high school mark) and 
selecting nine school ALSERUCES And ie SChOOLs for alIUstrative 
purposes, scores were computed for sub-groups identified on the basis 
of the selected student characteristics. 

The results revealed differences among schools and districts 
Bangangerrom 314 tO 784 00f the students achieving a high school 
diploma overall, and additional differences upon an examination of the 
various student sub-groups. For example, some schools showed 
achievement rates higher for males than for females, others exhibited 
notable differences from the provincial norm for the low achievement 
group, or in another instance, the work bound student. 

A further disaggregation showing both enrollments and percentage 
achieving a diploma for the three prior achievement groups within each 
of the four student programs was computed. Markéd differences from 
one school district to the next were revealed, which may have been the 
result of certain enrollment policies or differences in program 
effectiveness within the districts. 

The next chapter provides an overall summary of the study along 


with conclusions and implications. 
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CHAPTER | 


SUMMARY, CONCLUSIONS, AND IMPLICATIONS 


This chapter presents a summary of the study, some conclusions 
and several, implications. INBene tirnaleparee of the chapeer, a number 


of recommendations for further research are identified. 


Summary 

The research was exploratory in the field of macro indicators of 
educational effectiveness. The major area of interest was the 
identification of procedures for evaluating on-going school programs. 
The need for more information regarding the quality of schooling 
generally in the province of Alberta and, more specifically, the 
quality of education within general curricular areas and within major 
institutional units was the major concern of the study. The focus was 
on the provision of summary information of a quantitative nature, 


Bather Chan Gescripeive Qualitative information. 


Purpose and Problems 

The purpose of the study was to test a model for monitoring an 
educational system. It was intended that oe a monitoring approach 
would provide information on the quality of education in various 
program areas and at system and subsystem levels thereby providing 
guidance for curriculum policy changes and information for decision 
making on resource allocations. 


The problem involved the identification of appropriate variables 
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and an illustration of several ways in which such variables could be 

used in fulfilling the purposes outlined. The sub-problems adopted 

for the study were as follows: 

is to identify general and subject area indicators of quality 
education, 

Pig to identify student-based and school-based variables which 
account for significant variance in the indicators of quality 
education, 

Se to determine, for selected indicators of quality education, the 
total variance which is accounted for by the student-based and 


school-based variables and the explanatory power of each of these 


variables, 

4, to compute provincial norms for each indicator of quality 
education, 

aie to identify, for selected indicators of quality education, 


aifferences among groups of students classified on the basis of 
student programs and student characteristics, 
Gite to develop profiles for selected schools and districts using the 


various indicators of quality education. 


Conceptual Framework 

A variety of concepts which can be used to describe an evaluation 
perspective, along with the potential contributions of curriculum 
evaluation theory and other related fields were reviewed. A 
particular evaluation perspective was outlined, and a model which 
could be used when monitoring on-going school programs has been 


presented. 
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The particular perspective that has been outlined is a macro 
perspective of program evaluation, which is evaluation of a "gross" 
nature at "general" institutional or curricular levels, using quality 
indicators as the major informational tool. Guiding principles for 
the identification of these indicators of educational quality 
included: 

a. an emphasis on curriculum goals and objectives, 

b. brief summary statistics for the policy level decision maker, 

Cc. an inductive inquiry approach, 

d. accountability to the public a prime objective, 

e. provision of information without judgements, 

fe focus: On the provincial and schoo] district levels of 

organization, 

g. an emphasis on improving discriminatory capabilities in 

identifying "programs" taken by students, 

h. careful adherence to variable categorizations, 

i. a recognition of related evaluative disciplines such as needs 

assessments, organizational effectiveness and information 


systems. 


The conceptual model presented, proposed a three phase evaluation 


approach with these phases to be used in a cyclic manner. The three 
phases are: (a) the data collection and preparation phase, (b) the 
inductive inquiry phase, and (c) the information phase. The present 
study used existing data, hence the focus was on phases b and c. 
Three steps were outlined for each of the two phases. These were the 
source of the six sub-problems of the study. The steps describe the 


procedures to be followed in taking the variables collected, 
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hentaens potential quality indicators from these variables, and 
determining which of the antecedent and transactional variables are 
most associated with variance in the quality indicators. 

The model also portrays the close relationship that program 
monitoring at the macro level has with more in-depth needs 
assessments, educational research, curriculum evaluation, 
organizational effectiveness studies, and the general social indicator 


movement. 


Methodology 

The methodology differed for the first three and the last three 
sub-problems of the study. In the former case, the inductive inquiry 
phase of the study, exploratory and analytical statistics were used, 
and comparisons made of the various results in order to identify which 
variables might have utility as indicators of educational quality. 

For the remaining three sub-problems which were associated with the 
information phase of the study, use was made of descriptive statistics 
only in order to present the quality indicators in such a manner that 
the loci of differences in these indicators might be identified at the 
various institutional and program levels. 

The outcome variables used in developing the indicators of 
educational quality included official Department of Education goals 
and objectives, accepted achievement indicators within the schools 
such as marks and credit achievements, and career outcomes such as 
post secondary education, occupational status, etc. 

Specific methodologies used for each of the six sub-problems 


were: 
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the Identifying quality indicators. Outcome variables, where 
Pent goal referenced, were examined for potential overlap 
using intercorrelation data and thoughtful examination. 

are Identify explanatory variables. A two step procedure was 
utilized which included a statistical procedure called SEARCH 
followed by analysis of variance. 

Bic Relative explanatory power. Two approaches were used for the 
purposes of comparison, both depended upon Multiple 
Classification Analysis. One approach provided overall multiple 
correlation coefficients with beta ratings of the relative 
contribution of each independent variable. The second approach 
computed the "marginal" explanatory power of each independent 
Variable. 

4, Identify provincial norms. Computation of means was done where 
these were appropriate, and in other instances percentages were 
compared. 
according to students Characteristics. Crosstabulations and 


percentage comparisons were used. 


—_—_-_ ---oeoeoeororormnaS Oe ee, 


districts and schools. Crosstabulations and percentage 


comparisons were used. 


Results: Inductive Inquiry Phase 
Following are the major findings associated with each 


sub-problem. 
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Sub-problem i. General quality indicators identified were of two 
types: internal and external. The main internal indicators related to 
achievements within school were: 

Ser Percent -achievingra highschool diploma; 

See lehCone AGChLeVing NakerculacLon. 

- Average of high school marks. 

The external indicators identified, concerned career achievements 
as reported five years after school, and the ratings of i2 goals of 
education. The career achievement indicators were: 

- Average years of further education, 

—- Percent unemployed and seeking a job, 

- Average Pineo-Porter job status level, 

- Percent earning over $12,000 annually, 

- Percent Satisfied/very satisfied with their career. 

The goal ratings indicators were: 

- Percent rating adequate or very adequate: "Basic Competencies" 

- Percent rating adequate or very adequate: "Use Information" 

- Percent rating adequate or very adequate: "Communication 

Skiadis" 
- Percent rating adequate or very adequate: "Health, Fitness, 
Safety" 

- Percent rating adequate or very adequate: "Respect Others" 

- Percent rating adequate or very adequate: "Desire Learning" 

- Percent rating adequate or very adequate: "Understand Changes" 

- Percent rating adequate or very adequate: "Good Citizenship" 

- Percent rating adequate or very adequate: "Leisure Time" 


- Percent rating adequate or very adequate: "Appreciate Culture" 
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- Percent rating adequate or very adequate: "Management Skills" 
- Percent rating adequate or very adequate: "World of Work" 
Subject-area indicators identified for each subject area were: 
Exeent of Student participation in each subject area. 

Seventeen indicators (#1-#17) concerning average credits per 
student in subject areas were identified: Matriculation English, 
General English, Social Studies, Social Science, Matriculation 
Mathematics, General Mathematics, Matriculation Science, General 
Science, Second Languages, Art, Drama, Music, Physical Education, 
Business Education, Home Economics, Industrial Arts, Vocational 


Education. 


Another 17 indicators (#18-#34) concerning percentage ‘of students 
taking one or more (1+) courses in subject areas were identified: 
Matriculation English, General English, Social Studies, Social 
Science, Matriculation Mathematics, General Mathematics, 
Matriculation Science, General Science, Second Languages, Art, 
Drama, Music, Physical Education, Business Education, Home 


Economics, Industrial ArtsS, Vocational Education. 


An additional 17 indicators (#35-#51) related to the average 
number of courses taken by those involved in subject areas. 
These were: Matriculation English, General English, Social 
Studies, Social Science, Matriculation Mathematics, General 
Mathematics, Matriculation Science, General Science, Second 
Languages, Art, Drama, Music, Physical Education, Business 


Education, Home Economics, Industrial Arts, Vocational Education. 
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Marks achieved by students in each subject area. 

Seventeen indicators (#52-#68) related to the average mark 
achieved by those involved were computed in each of the following 
subject areas: Matriculation English, General English, Social 
Studies, Social Science, Matriculation Mathematics, General 
Mathematics, Matriculation Science, General Science, Second 
Languages, Art, Drama, Music, Physical Education, Business 


Education, Home Economics, Industrial Arts, Vocational Education. 


Grade distribution in each subject area. 
Another 17 indicators (#69-#85) concerned the percentage of A and 


B grades by those involved in subject areas: Matriculation 


English, General English, Social» Studies, Social Science, 


Matriculation Mathematics, General Mathematics, Matriculation 
science, General Science, Second Languages, Art, Drama, Music, 
Physical Education, Business Education, Home Economics, 


Industrial Arts, Vocational Education. 


One more set of 17 indicators (#86-#102) identified the 


percentage of D and F grades achieved by those involved in 


Subject areas: Matriculation English, General English, Social 
Studies, Social Science, Matriculation Mathematics, General 
Mathematics, Matriculation Science, General Science, Second 
Languages, Art, Drama, Music, Physical Education, Business 


Education, Home Economics, Industrial Arts, Vocational Education. 


Evaluation of the achievement of objectives within each subject 


abeay. 
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An average adequacy rating (#103-#114) was computed for each of 
12a subject' areas! English, Social Studies, Social Science, 
Mathematics, Science, Second Languages, Art, Drama, Music, 


Physical Education, Business Edvcation, Industrial HMaucation: 


Sub-problem 2. One-way analysis of variance was undertaken for 
each of the independent variables using selected quality indicators as 
the dependent variables. Generally, the relative predictability of 
the independent variables did not differ from one dependent variable 
to the next. Based on the extent of variance in the outcome measures 
accounted for, seven of the independent variables were selected for 
further analysis. These seven were student program (v54), 
pupil-teacher ratio (v58), sex of respondent (v63), T-SCAT (v66), 
grade nine average mark (v73), self ability rating (v74), and career 
Plans (v78). 

supb-probiem 3.. The analysis revealed that over 50 percent of the 
variance in high school achievement level and in the average high 
school mark was accounted for by the five predictors 
(v54,v88,v63, v66, 73) selected for analysis. Predictability of 
subject area outcomes varied from high in matriculation English to a 
low in Music. Both approaches demonstrated the Significant degree of 
association between the program that the student follows in high 
school and the achievement of most of the internally measured 
outcomes. To a lesser extent, but more than for other predictors, the 
same was found for the externally measured indicators of quality 


education. 
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Results: Information Phase 

The information phase of the study was primarily concerned with 
the presentation of scores on the quality indicators in a form that 
would help policy makers and curriculum planners gain a general idea 
of the quality of education in their area of interest. Sub-problems 
4, 5, and 6 associated with this phase focused in turn on (a) the 
provincial scene, (b) various sub-groups of students based on 
antecedent characteristics or school program, and (c) various 
representative organizational units. The findings for these 
sub-problems were as follows: 

sub-problem 4. Two types of provincial level quality indicators 
were identified. The first of these focused on "schooling" generally 
and the second on each of 17 subject areas atthe general level. Some 
of each of the two types were based on school data (internal), and 
others on follow-up data (external). 

For the general quality indicators, provincial norms based on the 
particular sample data used, were computed. These norms were: 
Ue Internal 

-— Sixty-five percent of students achieved a high school diploma. 

- Thirty percent of students achieved matriculation status. 

- The average high school mark over all courses was 61.6%. 
2 External (5 years after high school) 

- Average years of further education was 1.7. 

- Average unemployment rate was 5.8%. 

- Average job status level (Pineo-Porter) was 46. 

- Thirty-one percent reported earning $12,000 or more. 


- Sixty-four percent reported being either satisfied or very 


satisfied with their career. 
- Overall, the goals of education were rated adequately or very 


adequately achieved by 55% of respondents. 


These norms present the provincial picture for a particular year. 


They provide a base for comparison with sub-groups within the sample 
or for comparison with similar samples in other years. 

Subject area quality indicators yielded provincial norms showing 
(a) the level of student participation, (b) the level of student 
achievement, and (c) student ratings of the achievement of objectives 
for each of the 17 Subject areas. 

In Figure 3, three selected quality indicators are presented in 
profile format based on data taken from Table 9. The profile on 
student participation in each subject area graphically displays the 
differences from subject area to subject area in the percentage of 
students taking one or more courses; the profile on level of 
achievement presents at a glance, comparisons in the percentage of 
students receiving a grade of either D or F in the 17 subject areas; 
and the profile on rating of subject area objectives presents a 
Similar graphical representation for the third variable. The 
following generalizations can be drawn from these profiles: 
lie Participation: 

- the percentage of students taking one or more courses in each 
of the subject areas ranged from 99% in Social Studies and 
Physical Education, to lows of between 10 and 20% in Music, 
Art and Drama; 

- general courses in English, Social Studies, Mathematics, and 


Science enrolled considerably fewer students than did their 
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COMPARISON OF SUBJECT AREAS ON THREE QUALITY INDICATORS 
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PERCENTAGE 


matriculation counterparts; 

- Home Economics, Industrial Arts, and Vocational Education 
subject areas each enrolled about 25% of the students who took 
at least one course in the subject area. 

Pee, Achievement: 

- the highest percentage of students receiving grades of D or F 
was found in the general Mathematics and general Science 
courses, 

- low grades were less prevalent in English courses than in 
Social Studies or Social Sciences, 

- the fewest D and F grades were found in Drama, Music ana 
Physical Education with Music being the lowest of the three. 


as Objective ratings: 
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-— subject areas with the highest ratings on the adequacy with 
which they were deemed by ex-students as meeting their 
objectives included Physical Education, Matriculation 
Mathematics and Vocational Education, 

- the Second Language program objectives were considered to be 
less adequately achieved than were those for the other subject 


acedas. 


Sub-problem 5 focused on a more detailed examination of the 
relationship between each of sex of respondent, prior achievement 
level, and student program on the one hand and the quality indicators 
on Pee other. These three variables, based on student 
Characteristics, had earlier been identified as being associated with 
the greatest amount of variance in the quality indicators. 

Figures 4 and 5 present in graphic form, the general quality 
indicators differentiated by sex of the student. Figure 4 reveals 
that over 10% more females than males achieved a high school diploma. 
Also displayed is the higher percentage of females attaining 
matriculation status and higher average high school grades. In terms 
of employment level, no difference is evident in the unemployment 
rate. Females are shown as holding higher prestige jobs on the 
average, but a greater percentage of males received wages of $12,000 
or more per year. The level of job satisfaction appeared to be about 
the same for both sexes. 

A somewhat greater percentage of females than males rated the 
general goals as adequately or very adequately achieved. A more 


detailed examination of the ratings for individual goals is presented 


1158S. se ad 


ay f : 


We 


ea 

a? Qeoo tote @eaeraeege 
; : ; ‘9 7» =a 3 
oC” iMG © Saath ‘epee 


; ; j _ 
OC cal: Sete ati | 


wey 


vp ite. UL ape Fhe, 


a) i a) 
+ 4a ‘ 
ae fn 
a» 
‘ 
> oes eS 
© ~ 
; Lets ge 
\ 
: ' ) “9 
i . | he 
Be 
‘ é 
4 ‘ a ie 
ind 
* 5 ' 4 - 
1 4 ‘ SLA We haw 
t 
oi 
ii “ } 


‘oe ht ro ena? 
~ at .0jo) Sey ogee 
2 1 oe 


a Sons 9 hoe! ams ya 


a of xe os a -_ 
wy 4 tes + 409), Aa 
ra an 


PERCENTAGE 


GOAL 1 
GOAL 
GOAL 
GOAL 


2 
3 
4y 
GOAL 5 
GOAL 6 
GOAL 7 
GOAL 8 
GOAL 9 
GOAL 10 
GOAL 11 


GOAL 12 


100 


Figure 4 


GENERAL INDICATORS - BY SEX 
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RATINGS ON GENERAL GOALS - BY SEX 
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in Figure 5 which shows a greater percentage of females than males 
rating the level of goal achievement as adequate or very adequate for 
each goal except Goal 2, "use information." The difference is 
greatest in the case of Goal 12 which pertained to preparation for the 
world of work. 

Figures 6 and 7 focus on possible differences in the indicators 
of general quality based on different levels of prior achievement by 
the students, utilizing their grade nine average. Differences of 
close to 20% in the percentage of students attaining a high school 
diploma are evident between the low and middle and also the middle and 
the high prior achievement groups. Even larger differences are 
evident in the percentages of each group attaining matriculation 
status, with only 20% of the middle group and three percent of the low 
prior achievement students gaining matriculation. Differences among 
the three groups, although in the same direction, were not as great 
for average grades achieved throughout high school. Occupationally, 
there was a lower rate of unemployment for the middle prior 
achievement group than for either of the other two groups. Job status 
level increased somewhat from the low through the middle and the high | 
prior achievement groups but all three groups were about the same on 
Career Satisfaction. Differences among the three groups for the 
remaining variables in Figure 6, namely, wages, and goals were 
minimal. 

Overall, the achievement of the general goals was rated somewhat 
lower by the high prior achievement group of former students. 
Differences among the three prior achievement groups, in the average 


percentage of respondents rating the goals of education having been 
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Figure 6 


GENERAL INDICATORS - BY PRIOR ACHIEVEMENT 
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adequately ys very adequately achieved, varied from goal to goal as 
evident in Figure 7. Goals rated higher than the norm by the high 
prior achievement students were often rated lower than the norm by the 
low achievement students, and vice-versa, with the middle achievement 
group falling somewhere in between. The high group had higher 
adequacy ratings for goals 1, 2, and 6 dealing with the development of 
basic concepts, using information, and creating a desire for learning. 
On the other hand, they had the lowest adequacy ratings for goals 4, 
feeds) 42, Its endet2 dealing@with individual and: social well-being and 
work preparation. The middle group had ratings higher than the norm 
for all goals except goals 2 and 3, dealing with learning to use 
information and developing communication skills respectively. The 
development of management skills (goal 11) was rated notably higher by 
the low prior achievement group than either the medium or high groups. 
Figures 8 and 9 illustrate the differences among the four student 
programs with csctey he to the general indicators. For the high school 
diploma variable, percentages recorded were highest for students from 
the Matriculation program and lowest for students from the General 
program. Few students achieved matriculation status other than those 
enrolled in the Matriculation program. Average high school marks 
followed the same pattern as for the diploma variable but with the 
differences considerably less. Not shown in the graph, the number of 
years of further education, indicated students from the Matriculation 
program reporting the largest number, with Business Education students 
reporting the lowest and students from the other two programs on 


average falling about half-way between. 
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Figure 8 


GENERAL INDICATORS - BY STUDENT PROGRAM 
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In relation to the unemployment variable, both Business ana 
Vocational students showed zero unemployment with the highest rate 
registered for the General program. Job status level (Pineo-Porter 
ratings) was highest for graduates from the Matriculation program, 
followed by the Business, the General and the Vocational programs. 
Students from the Vocational program however reported a notably higher 
wage level, with Business students reporting the lowest wages and the 
Matriculation and General students being approximately equal and 
falling midway between the Business and Vocational groups. Career 
satisfaction was highest for students from the Vocational program, 
followed by those from the Matriculation, General and Business 
programs. ~ 

Overall, the adequacy of educational goal achievement was rated 
highest by students from the Business Education program, followed by 
those from the Vocational Education program. Figure 9 presents in 
profile form the ratings by program for each of the 12 goals. The 
profile for the Business program displays ratings on the 12 goals 
higher in most instances than in the other three programs. In the 
case of goal 3 (communication skills), goal 11 (management skills), 
and goal 12 (world of work), the ratings by the former students of the 
Business program were notably higher than was the case for the other 
programs. The profile for the Matriculation program displays ratings 
considerably lower than the other three programs in the case of goals 
85°95 10;°11, and) 12. (The) Generalvprogram profile displayseratings 
higher than the Matriculation program on goals 8 to 12, but lower or 
equal on goals 1 to 8. The profile for the Vocational program falls 


midway among the others, usually higher than the Matriculation and 
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General programs but lower than the Business program. 

Sub-problem 6 focused on the quality of education within selected 
institutional units. Six geographical regions of the province were 
examined along with nine selected school districts/counties and 12 
illustrative schools chosen to be representative in terms of size and 
geographic location. Regional differences were minimal and for this 
reason are not reported in detail in this summary chapter. On the 
other hand, differences among the school districts as well as among 
the schools chosen for comparison were quite marked. These 
Settecrencessare aL livstrated in Figures 10 “and: 1/1). 

In these two figures, the percentage of students achieving a high 
school diploma from four of the school districts and three of the 
schools selected from the larger groups of districts and schools are 
presented, based on the computations reported earlier in Table 25. 
District 4 showed notably higher percentages achieving a diploma (13% 
overall), especially in the case of the females and the low and middle 
aie achievement groups. Much lower percentages for all groups 
Within District 2 achieved a diploma. Comparisons between districts 6 
and 7, reveal a great Similarity between them with one exception: for 
the low prior achievement group, the percentage achieving a diploma is 
about 104 “Lower jin District 6. 

The schools comparison in Figure 11 reveals that for both males 
and females, School 7 had close to 40% fewer students receiving a high 
school diploma than did either of the other two seine: ne 
difference was not quite as marked in the case of the three prior 
ability groups. Further comparison between the male and female 


profiles reveals that School 8 differs from the other two in the 
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Figure 10 


PERCENT ATTAINING HIGH SCHOOL DIPLOMA CLASSIFIED BY SEX 


AND PRIOR ACHIEVEMENT FOR FOUR SELECTED DISTRICTS 
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Figure 11 


PERCENT ATTAINING HIGH SCHOOL DIPLOMA CLASSIFIED BY SEX 
AND PRIOR ACHIEVEMENT FOR THREE SELECTED SCHOOLS 
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relative distribution between the two sexes. School 8, in contrast to 
the other two schools and to the provincial norm, had a lower 
percentage of females than males achieving a diploma. Profiles for 
the three prior ability groups are very Similar with the exception of 
the low prior ability group which is the only one where School 6 has a 
lower percentage of graduates attaining the high school diploma than 
School 8. 

Comparisons of districts on their average high school grades 
assigned showed differences much less pronounced than when comparing 
the achievement of a diploma. District 4, which showed high 
achievement on the diploma indicator, did however have somewhat higher 
grades as well. Overall averages recorded for the selected schools 
showed a spread of less than five percentage points. Differences 
larger than this however were recorded for some of the prior 
achievement groups. Table 26 may be referenced for more detail. 

For the purposes of illustrating in summary form the findings 
from the previous chapter, particularly those contained within Table 
27, two additional figures, Figures 12 and 13 were devised. Figure 12 
presents a comparison of the percentage of students attaining a high 
school diploma, focusing on prior achievement groups within each of 
the four student programs. In order to facilitate comparison between 
diploma profiles and enrollment profiles for the same two districts, 
Figure 13 was devised. 

In the Business Education program, as Figure 13 reveals, 
enrollments were very Similar for all three prior achievement groups 
in both districts, .with most students coming from the low prior 


achievement group and very few from the high. In contrast, as Figure 
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Figure 12 
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Figure 13 


ENROLLMENT WITHIN PROGRAM BY PRIOR ACHIEVEMENT 
FOR TWO SELECTED DISTRICTS 
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12 reveals, District 7 had higher percentages of students complete a 
diploma from the low and middle groups, but fell far behind District 6 
for the high achievement group. 

in’ the Vocational program, as Figure 13 reveals, District 7 
enrolled higher percentages of the low and middle prior achievement 
Students and fewer of the high. Figure 12 shows notably higher 
percentages (20% to 40%) of the students within the Vocational program 
acn+evVing a diploma in District 7 than in District 6. 

As Figure 13 reveals, fewer students were enrolled in the 
Matriculation program within District 7, especially from the lower 
achievers. Figure 12 however, reveals the percentages achieving a 
diploma were higher than District 6 for the low achievement group but 
less for the middle and close to the same for the high prior achiever. 

The General program enrollment percentages were higher for all 
three achievement groups in District 7, as revealed in Figure 13. 
Comparison of the profile in Figure 12 however, shows what appears to 
be a consistent finding over all four programs, namely, a higher 
percentage of lower achievers completing a high school diploma. 

The primary purpose of the analysis and report of findings 
associated with sub-problems 4, 5, and 6, of the study was to 
illustrate the types of information which could be gleaned from the 
macro evaluations using the various indicators of educational quality. 
For this reason, only selected analyses were undertaken and selected 
findings reported in the previous chapter and in this summary chapter. 
A detailed presentation of all the possible analyses which could have 


been undertaken in the present study was not intended. 
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Conclusions 


Conclusions have been identified for each of the two phases of 


the study. 


Inductive Inquiry Phase 


The following points are presented with reference to the 


inductive inquiry phase: 


ae 


The list of indicators of educational quality identified in the 
present study, while perhaps incomplete in the ideal sense of 
being mutually exclusive yet comprehensive, represents an 
important step toward this ideal because of the low statistical 
overlap among indicators and the theoretical soundness of them. 
The school program chosen and hence the type of school experience 
undergone by students would seem to have a significant impact on 
achievements within school as well as on later career 
achievements. 

Sex of the student was found to be associated with in-school 
achievements and, to some degree, with career outcomes. 

Student ability was closely associated with both internal and 
external outcomes of schooling, but prior achievement scores 
proved to be better predictors than did the more ability 
orientated T-SCAT scores. 

Many of the features of schools which are generally assumed to be 
related to educational effectiveness such as pupil-teacher ratio, 
school size, types of programs offered, etc., were found to have 
low association with variance in the quality indicators examined 


in the present study. 
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With the use of the theoretical model selected for the study and 
by focusing on only the most promising antecedent and 
transactional variables, significant levels of association were 
FOURATWLEEn Eherqualt ey eindicacons dence taed 7s huGcner “esearch 
and sophistication of the techniques utilized shows promise of 


worthwhile results. 


Information Phase 


The following points were identified with respect to the 


information phase of the study: 


bts 


Significant numbers of students were still not completing 
secondary EGE Gn (352%).4an 1973. 

Goals associated with the "back to the basics" curricular areas, 
such as the development of basic competencies, use of 
information, and communication skills (goals 1, 2, and 3) were 
among those evaluated as having been most adequately achieved. 
The needs of the low achievement student appear to be 
inadequately met (low diploma completion rates, low credits 
achieved, low participation in subject area offerings) except for 
those few students who follow either a Business or Vocational 
program. The Mathematics and Science courses showed particularly 
low ratings in reference to the low achievement group. 
Enrollments in the Fine Arts courses appear to be very low which 
may account for the overall low evaluation of the educational 
goals which tend to reflect these subject areas. 

Though few students followed a Business Education program, a 


large number of students took at least one course from those 
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offerings. 

S. The Physical Education curricular area stands out from the other 
subject areas in the number of students it serves, with 
relatively few failures, and high evaluative feedback regarding 


the achievement of subject area objectives. 


Summation 

Based on the above more specific conclusions, a major conclusion 
can be stated for this exploratory research, namely, that macro 
evaluations of school programs of the type demonstrated by this study 
have potential in providing useful information for school program 
planners and educational decision makers at the general level. While 
the importance of the specific findings of the study may be questioned 
because they are time specific, the periodic availability of the type 
of information identified could be useful in assisting the educational 
decision makers to improve the general quality of education. An 
additional advantage of this approach is the continuing dialogue with 
the public, as ex-clients of the schools and present financial 
Supporters. Public information regarding the quality of education in 
various respects, provided on a periodic basis would be one approach 
to offsetting unfounded and biased speculation which is often advanced 


in the absence of systematically collected data. 


Implications 
Possible implications of this study have been identified with 


reference to practice, theory, and research. 
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Practice 


The following implications are identified with respect to 


practice: 


ilies 


If curricular goals and objectives are to be used as a structure 
for feedback on the quality of education, curriculum planners 
need to continue refining the wording of these statements to 
ensure a common understanding of the meaning by both the 
professional and the lay person. 

School administrators need to examine more closely their policies 
and procedures in packaging groups of courses for students and in 
counselling students into certain programs, in order to ensure 
that student programs (groups of courses) are available to the 
type of student who appears to benefit from them. 

A high school program may be an entirely different set of 
experiences for one student than for another; therefore 
evaluators of schools need to differentiate evaluative feedback, 
collected from students and the community, based on the type of 
student program being referred to by the respondent. The 
importance of such differentiation has been demonstrated by this 
study through showing the degree of association of the student 
program variable with variance in the indicators of quality 
education. 

There would seem to be a need for school administrators to focus 
more on the components of their school offerings experienced by 
students, and less on traditional institutional characteristics 
such as pupil-teacher ratio. 


School curricula need to be improved particularly in the areas of 
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preparation for work, leisure time, appreciation of culture, and 
gooa citizenship. Curricula in the areas of the development of 
basic competencies, communication skills, and how to use 
information, are strong in relation to the other goal areas. 

In light of the low ratings on the adequacy of objective 
achievement, an in-depth evaluation would seem required of the 
Second Languages program, including a close examination of the 
Viability and understandability of the present objectives. 
Curriculum evaluation would seem to be required in the "generai" 
courses, particularly in Mathematics and Science, which seem to 
be serving fewer students than, for example, the general English 
courses, and awarding greater percentages of low and failing 
grades to those who do enroll. 


Examination of grading practices throughout the province may be 


advisable, in light of the aforementioned high percentages of low 


grades awarded in Mathematics and Science and those in Social 
Studies as well. At the other end, by contrast, extremely high 
average marks were evidenced in the Music area. 

A very critical examination of the General program is required, 
especially for the low achievement student. Schools need to 
determine why students follow this route, why it is not meeting 
their needs as evidenced by the low percentage of completions, 
and what alternatives should be considered in order to better 
meet the needs of this large group of students. 

More emphasis needs to be placed on providing expanded Business 
and Vocational programs, especially for the low prior achiever, 


as the findings have shown a combination of high completion 
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rates, high Settee on external indicators, and high ratings on 
the achievement of objectives by students enrolled in these 
programs. 

11. Closer attention needs to be paid to the quality indicators 
associated with individual school units, including enrollment 
practices and program outcomes, in order to identify units which 
May require more indepth evaluation to determine why there 
appears to be a low quality of education provided. Close 
examination of those units with high ratings on the quality 
indicators may also reveal factors which should be considered for 


adoption by other school units. 


Theory 

The implications of this study for evaluation theory generally 
fall in two main areas, First, the use of the inductive inquiry 
phase, identifying independent variables which show the most potential 
for examination in greater detail, has shown some level of success and 
requires further development. Second, the considerable variance 
accounted for by the student program variable suggests an increased 
emphasis on identifying the process components in any program 
evaluation, where the components may differ, and disaggregating the 
outcome data based on the categorizations that may be developed from 
such examination. While most evaluation models that are based on 
systems theory do identify process as one focus of evaluation, further 
development of this area should result in fewer oversights in the 
application of these modeis. 


The model for monitoring an educational system which was tested 
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by this study, requires further development in at least two major 
areas. While the data base ae an educational monitoring system must 
be based on the student, as the user of the system, nevertheless 
evaluative sources must be expanded to include the professionals 
involved as well as the parents of the students. ineaddateronm., sehe 
indicators developed were either averages or percentages. Further 
development of the model should explore the combination of two or more 
Variables, producing "rates" and "ratios'' as a second level set of 
indicators. The identification of areas requiring development would 
result from : second similar attempt, which would identify areas of 
theoretical concern as a result of discrepencies in the findings of 
two comparable studies. Attention also needs to be directed toward 
the possibility of more explicitly recognizing the place of macro 
qualitative data of a descriptive nature, which may be required to 


supplement the quantitative emphasis provided in the model. 


Research 
The following implications for further research are identified: 

Lhe Further research needs to be undertaken on the question of 
whether variables other than sirens (i.e. antecedent or 
transactional) could be included as quality indicators if they 
were shown consistently to be associated with desired outcomes. 
Such findings may alter the initial position adopted within this 
study that only outcome variables be considered as indicators of 
quality education. 

ie The quality indicators identified in this study need to be tested 


by replication and by comparison with other indicators of 
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educational quality. Further research should seek to add 
additional indicators not covered in this study. 

Further research needs to focus on the definition of student 
program identifying the curricular components of "Schooling" as 
experienced by different groups of students. 

Since the characteristics of schools used within this study did 
not show close association with any of the quality indicators, 
yet schools and even school districts were shown to be associated 
with variance in the indicators, further research needs to be 
undertaken in order to identify the features of the schools and 
school districts which may contribute to the differences in the 
quality of education. 

There is a need to emphasize research with regard to the long 
range impact of educational programs. Few of the antecedent or 
transactional variables included in this study showed measurable 
amounts of association. Indeed, even the quality indicators 
(outcomes) taken from school records showed low levels of 
association with the career outcome measures or the ex-student 
evaluation of goal achievement. This would seem to indicate that 
the current measures used within the school to evaluate the 
progress of students do not coincide with the achievement of our 
stated goals, at least as envisaged by our former students. 
On-going, periodic research is required to update provincial 
norms and identify potential trends for the quality indicators 
identified in this study, and to identify additional indicators 
of Quality education: 


In-depth research iS required in order to establish the nature of 
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the relationship between student evaluations of course objectives 
and the more objective measures of skill development and 


knowledge attainment within the subject areas. 
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Listing Of CEP Variables 


From Departmental Records: 


Variable 
V1 
V2 
V3 
V4 
V5 
V6 
V7 
v8 
v9 
V10 
Vii 
V12 
Wats 
Vi4 
V15 
Vi6 
V te? 
V1i8 
v19 


From Questionnaire 


Variable 
V20 
V21 
V22 
Vi23 
V24 
Vi 5 ae 
Wale. = 
V47 


y5oe— 
Yoo. 
vso — 
Vor 


ViO2 = Viade 


Viitoe Vics 
V124 - V134 


Vi35 
V136 
Viz 
V1i38 
V139 
V1i40 


V141 
V142. 


V35 
V46 
V57 


V68 
V79 
Vso 
V101 


Description 
Sex 
Grade 9 V-SCAT 
Grade 9 O-SCAT 
Grade 9 T-SCAT 


Grade 9 Departmental Average 

High School Credit Attainment 

Grade XI Program - 4 way Categorization 
Grade XI Program - 15 way Categorization 
Grade X Average 

High School Average 

Grade XI School - Total Credits Offered 
Grade XI School ~ Enrollment 

Grade XI School - Number of Teachers 
Grade XI School - School Type 

Grade XI School - Rural/Urban/Suburban 
Grade XI School = School Code 

Grade XI School - Pupil-Teacher Ratio 
GradewxXa Schools No of Counselors 
Grade XI School - Type of Term 


One: 
Description 
Age 
Father's Education 
Mother's Education 
Intelligence Self-Perception 
Courses chosen 
Course Pattern Requested 
Reasons For Course Pattern Requested 
Course Patterns in Which Enrollment 
DigeNoEVOCcuE 


Reasons For Not Enrolling In Marked Courses 


Who Influenced Course Pattern Choice 
Reasons For Dropping Courses 

Extent Of Course Contribution To 
Understanding Occupational Activities 
After Leaving School 

Extent Of Course Contribution To 
Preparation For Leisure Time Activities 
Course Choices If Starting Grade XI Over 


Teaching Method Most Often Used (By Subject) 


Plans To Finish High School 

Reasons If Planning To Leave School 
Plans On Leaving School 

Who Influenced Decisions 


Parents Educational Plans For Respondents 


Parents Plans For Respondents After 
High School 

Does School Have A Guidance Counsellor 
Career Help Sought From 
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From Questionnaire 


Variable 
Vici ieee Vit ST 


Vvi88 - vi98 


Wiis VN 209 


V210 
W211 
V212 


V213 
V214 
V215 
V216 
V2i 7 
V218 


V2035 
V220 
Vacs VO 


V231 - V244 
V245 
V246 
V247 
V248 
V249 
V250 


From Questionnaire 


Variable 

eV 25a 
Vi252 
V253 


Importance Of Information Provided By School 
In Occupational Choice 

Importance Of School In Preparing Leisure 
Time ACELVIELES 

Importance Of School In Preparing 
Occupational Skills 

Career Decision Made 

Membership In School Organizations 
Membership In Out Of School Organizations 
Learning Activities Evaluation 

Evaluation Of Features In School 

Work Experience Course Credit 

Contribution Of Work Experience Course 

In Educacion 

Participation In Special Project For Credit 
Evaluation Of Special Project Experience 


Two? 

Description 
Contribution Of Course Patterns In 
Understanding Occupational Activities 
Contribution Of Course Patterns In 
Preparation For Leisure Time Activities 
Course Pattern Chosen If Starting Grade 
11 Again 
Finished Grade 12 
Plans For Finishing Grade 12 
Plans For Leaving School Before 
Completing Grade 12 
Importance Of Education 
Current Status 
Who Has Had Most Influence 
People Asked For Career Help 
Importance Of Information In Career Choice 
Importance Of School In Preparation For 
Leisure Time Activities 
Importance Of School In Occupational Skills 
Decision On Career 
Importance Of Learning Activities In Helping 
You Learn 
Area Ratings Of Last School Attended 
Importance Of Work Experience Course 
Importance Of Special Project Credits 
Time To First Job After Leaving High School 
DiPeLIculey el nMObDtaw in Gm helns cao 
Help in Finding Firse vob 
Reveteslenljoieslofel Iitg) (Syehbiohy Iie Ge) Waser 


Three: 

Description 
Educational Programs Involved In 
Education Since High School 
Current Occupational Status 
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V254 
W255 
V256 
Pa Mt 
V258 
V255 
V260 


V261 


V262 
V263 
V274 
Va75 
V276 
V310 
V373 
V381 


V273 


V309 
V372 
V380 
V444 
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Employment Type 

Status Of The Industry 

Current Wage Bracket 

Distance From High School Attended 
Satisfaction Of Career 

Time Elapsed Before First Employment 
Relationship Of Present Vocation 

To Previous Plans 

Reasons For Leaving School Before 
Completing Grade 12 

Career Help Sought From 

Courses Chosen If Starting Grade 12 Again 
Occupation Chosen If Starting Career Again 
Business, Service, Industry 

Achievement Of General Aims 

Achievement Of Subject Area Aims 

General High School Experience 

Importance Of Subject Area Aims 
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APPENDIX B: CEP Followup Questionnaire 


4 


In this first section we seek information about yourself since you left school. Please carcle 


the number opposite the appropriate response. 


1. What type of educational programs have 

you been involved in since leaving school? 
(Circle One Namber) 

NONE conc ceececce eiatalcietetels is (cieiale eccccce mralenl 

ONiverSity ccc ccc ccc ccc scceccccccece 2 

BEL OGGPcecce ccc cow se ccc scoeeucvesccceans 

Institute of Technology .--.--s-cccceceee eG 

Apprenticeship ..... 2.2. cee ewer eee eee eee 


Other Mts cisicia eclaiac ciao ceo cs ca ceesiecuces eG 


2. How many years of education have you 
completed since leaving high school? 
(Circle One Bumber) 
CGN SRS 00 COOHCOD CC OOCOCDOROODOOOCOROSS 
One year ADCO COO SO BROS CO OBO GOOOCO COSCO CER? 
Two years Peete ecco ce ccese cece wescees 
Three years Api SACI OE Oe 
Four OF MOFe years ..cce eee ew wee ce eee eee 5D 


3. What is your current occupational 
status ? 
(Circle One Number) 
Unea ployed: 
{a) actively seeking employment -------.1 
4b) not seeking employment Suaisislesiensieelet 
Student enn CRs eco cose ewseesecece Ss 
Employed «fill in blanks below) -------.4 
employed as a (job) 
in the business; 
indus*ry, or service. 
(include homemaking under erployed) 


e 
4. What is your current wage bracket? 
(Circle One Number) 
Up to $8,000 Ne aa cleo Cae a ae calm aait 
$8,001 *0 $12,000 ..cnceececcccerecceeee2 
$12,001 to $15,000 wcccccccceccecoseceen3 
$15,001 ANd OVE cccccceccccsocccesscce eS 


5. How far do you now live from the place 
where you received the major part of your 
high school education? 

(Circle One Number) 

0 to 25 niles CRI SSE OIC OOOO SABES 

25 to 100 mileS cc.cccccoce cs cccccccocce2 

100 to 200 niles EASES O DOOD OCC GORE: 

over 200 miles Pin Sect ace clea ceos een es oF 


6. How satisfied are you with-the present 
progress of your career? 

(circle One Number) 
Very satisfied See ecw ewe cose soccccocccse | 
Satisfied ..ccc-ca- rs eae vac om e1clcle eo wia't 
Moderately satisfied ABISISO OD CE OOOO GBS 
Only slightly satisfied soc cc ececescose eG 
Not satisfied at all .....e-eeeeceeeeeeeS 


7. How long from the time you left school 


did it take you to find your first 
employsent? 


(Circle One Number) 
0 to 1 month wecesceeece coer ccec= malclalatarent 
1 to 2 months AAO OOODOD CO BOOS MSS dan can7zs 
2 to 3 rsonths BOO HOODOO CI ODO COB CAD GOD E! 
re 
13 


3 to Bonths Ne atcisiss se\cle siecle oe slo sleleeleia ce. 
more than & monthS -«------ ecccccccccceeD 


8. Is your present vocation related to 


what you had planned during your high 
school years? 


(Circle One Number) 
Vere ee cccccadenuccecsicegeceos st 
Boderately sec cece ccc cne sc ceecceceee eZ 
Slightly PRE ee eet occtccdanaccemaaes sD 
Hot at all cccccccccoscccccccccecc ccs ee eS 


9. If you left school before completing 


grade 12, why? 


1 
+ 


(Circle One Busber) 
To get a job Neat aecctw ees ata ace wee! 
POOLE MATKSEL = eee a ccc cece os ceccececsenees 
To get married ER oe eines awe sin ceie ses 
Pinancial problems ....---cceereeeeeeeee4 
Dislike for school work ...------2e0- 2-5 


0. If you were still in school, which of 
hese people would you now ask for help in 


deciding a career? 


1 


«Circle One Nusber) 
Priends Pere tele cialacincls stcleleis claisicsals'ole.t 
Teacher eee cece ac ccaeseccccel 
Parents el 2 ce ot easeeaseces se6.5 
Guidance Counsellor ..-.cececcccccsoceee +4 
Principal 4 EE POOR CEET COOOL 
Other (Specify) AooosaoS 


1. If you could start grade eleven all 


over again, in which courses would you now 
register? 


1 
a 
A 


(Circle As Many Numbers As Necessary) 
Business Education -ccccccccccccccccceee ! 
Technical or Vocational Education ...-.-2 
Pnoglish ee oe cc ocicoceccascwmceaues 
Fine ALtS cceccccccccccccccccccccccccce eS 
Social Studies RI ASCO HIG Ge ODOR OIE) 
Home ECONOMICS ccccccccaccccc ccc ccccs ee ef 
Mathesatics’ © 2. tt ttiecesecees secant ager: 
Tndustrial ALCS sacccccaccesasccvesesesao 
COTORCE oe eee ce vees an eames ee gers 
Physical Education Slo ole ois ae cle ce slelsieciete alo 
Bodern Languages BSA ACA DOS HOA DOO OS OGM 


2. If you could start your work career 
gain, what occupation whould you chose? 


qjob) in the 
business, industry, Or 


ervice. 
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HOW ADEQUATELY DO ALBERTA HIGH SCHOOLS ACHIEVE THEIR GENERAL AIMS? 


This section presents some of the general “aims" that have been established for Alber*a high 
schools. {Not all of the aims are listed.) 

These aims, identified by the Department of Education, are general statements of what should 
be learned by students. 

How adequately did your high schoal program meet these general aims for you? 


Directions: 
1. Consider each aim listed below and circle the appropriate letters indicating your opinions 
of the adequacy of your education in regards to that ain. 


A A A 
D D Dap 
Ver E Nee OL K 
E Q Q OF ON N 
R U 0 Te 0 L! Xe) 
Yok A Ae Te W 
ZY Hh L 
E E F 
Develop *+he ability to use information and counselling services 
related to career deCcisSiONS. ses cccerccec cece cece ceseccrscccerrrerccs VA A NA DK 
Develop skills in managing natural, financial and human resources. --- VA A NA DK 
Develop an understanding of functions, responsibilities and 
achievements of various societal institutions. ---c-ssssresercrerrrerrns VA A NA DK 
Develop special interests and AbDiLItieS. cece cccc cece seccccccscreseece VA A NA DK 
Develop a positive attitude toward learning. .---ccceccccecces cesses VA A NA DK 
Develop an understanding of good physical and mental health 
practices. iOAOSIO GO COO COO OOD OO DO UDISOD IO OOO OOOO OS SOL SO OS MH Ts VA A NA DK 
Develop ability +o apply scientific methods in the pursuit of and 
analysis of knowledge. .----ecorcscccecccccssecccsrcssccsssscrcrscercs VA A NA DK 
Develop an attitude of respect for public and private property. ------ VA A NA DK 
Establish sound personal health habits. ..---ccesserrscerecrerrerceccs VA A NA DK 
Develop a feeling of cultural identity and heritage at national and 
international levels. ..-cccscccccescsscccscccccccsccssscsccescocccces VA A NA DK 
Develop a positive attitude toward participation in a range of 
leisure time activities - physical, intellectual and creative. ..-.--- VA A NA DK 
Cultivate appreciation for beauty, in various formS. ...-+++----eereeees VA A NA DK 
Develop an awareness of and the ability to adjust to a changing 
social and physical environment. ...ss-eeeeceeecreeecercscrcccrcesccnccs VA A NA OK 
Develop ability to adjust to the changing demands of Canadian 
society. LAD COR SOO SIO ODO IO OSES IO OE eR NOT I Rs cll de tM dl tl old VA A NA DK 
Develop an awareness of civic rights and responsibilities. ..-.--++s-- VA A NA DK 
Develop intellectual curiosity and eagerness for lifelong learning. -.. VA A NA DK 
Pstablish a good physical fitness Program. --++-eseeeeeeeeccercererces VA A NA DOK 
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HOW ADEQUATELY DO ALBERTA HIGH SCHOOLS ACHEIVE THEIR GENERAL AIMS? 


This section presents some of the general "aims" that have been established for Alberta high 
schools. (Not all of the aims are listed.) 

These aims, identified by the Department of Fducation, are general statements of what should 
be learned by students. 

How adequately did your high school program meet these general aims for you? 


Directions: 
1. Consider each aim listed below and circle the appropriate letters indicating your opinions 
of the adequacy of your education in regards to that ain. : 


A A A 
D D Daw 
VE E NoGE OL K 
E Q Q ORO anaN 
R U g Hel tof eB) 
Yara A A TWH 
el. ak ae 
E E E 
Develop understanding of the past, identify with the present and the 
ability to meet the future. ccc eecceceecee ce crrcccreccceccsccrccccce VA A NA OK 
Develop an understanding cf the Canadian and other forms of 
GOVEINMENK. soccer ccc c ccc s cers ccccccscccerscece cocccesccccccsccecccccce VA A NA DK 
Develop interests which will lead to a wise and satisfying use of 
leisure time. eececececenececececeeseeceecee ec eee eeceeeeeeneeeeeeeeeeeesaeeeeeresre eee VA A NA DK 
Develop an understanding of the obligation and responsibilities of 
Canadian and world citizenship. cccccccccccccce wcccccccceescesrcerccccs VA A NA DK 
Develop appreciation and respect for the worth and dignity of 
individuals. eceecececeeceoesceceocesceeoee ere eseeeceorseceeseeceoeeseeseeeeoec er eee eee VA A NA DK 
Develop an understanding of economic principles and 
CESPONSLDILIties. 22. e eee corse cree cere ces secs ccc cccescncrssscrcrrccccs VA A NA DK 
Develop creative self-expression *hrough various media incltding the 
fine and practical arts. cecccccsesceccccceeescesscccccccseseseseccecrs VA A NA DK 
Develop understanding and skills in the use of numbers, natural 
sciences, mathematics and social sciences. ...ceeeeeeeee reese rerceeeces VA A NA DF 
Develop skill in oral and written Languages. ccccccrccceecccccccccoccs VA A NA OK 
Develop a fund of information and CONnCePts. +. se-eereceereeeeerrrcreeees VA A NA DK 
Develop skills of thinking and proceeding logically. ....cecccccecoece VA A NA DK 
Develop ability in communicating ideas and feelings. ....cecce ec coeees VA A NA DK 
Develop special talents in the artS. eecsseereccsccccecoceecccsecccece VA A NA DK 
Learn to take into account the values of others when making personal 
choices. © 0 OOS OO OSS SS OES ESS SF SHO SS SCHHSSOHOEOSSE OOF ESE ODS SHHDEO VOSS OO SOLO VA A NA DK 
Develop skill in understanding the communication of otherS. --cccceces VA A NA DOK 
Develop ability to organize information into meaningful categories. .. VA A NA DK 
Develop skills basic to the world of work. Te cas ex Ne ee VA A NA DK 
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This section lists the "aims" that have been established for each subject area offered by 
Alberta high schools. 
These aims, identified by the Department of Education, are statements of what should 
learned by students in each subject area. 


How adequately did your high school program meet these subject area aims for you? 


Directions: 


hic 
2. 


Check each subject area in which you took at least one course. 
For each subject area checked, circle the appropriate letters 


indicating the adequacy of your education in regards to each stated 
ain. 


4) Art 


To 


To 


To 


To 


develop an appreciation of artistic deSign. 2.2.2 cee ee eee co cce ee 
develop an understanding of past and present art. ....---eceeeeeess 
apply artistic design principles in art and everyday living. .----- 


develop individual artistic skills. ......cccee eee eee ee eee cee eeeee 


4) Business Education 


To 


To 


To 


To 


To 


To 


To 


develop an understanding of the operation of a business. -.----+---- 
develop useful business knowledge and skills. .....---c---e eee eereee 
develop business skills of individual interest. ........-..----ee-- 


develop individuality in thought and action. ......--..22--eeeeees 


develop problem solving abilities. ... cece eee eens cece eee eee cceees 


apply business skills to other areas. 2.2... cscce ccc ce cers ceccces= 


give practical experience in the application of business skills. -- 


4{ L Drama 


To 


To 


To 


To 


To 


develop CONCENtLAatiONn. coc wcesccscccccrccccccccccccccccsscccccscccss 
develop sensory distinction. cc. cccce cece ccc cccccccccecccccccceece 
obtain freedom and control in physical movement. ......ccccecccoose 
encourage further interest in drama. ....cccewe cece cece ccecccocccccss 


communicate, through drama, an awareness of the current world 


issues. eeoeseeeeee een een eee eeeeeeeeeeeeeeeeeeseeseeveereseee ee eee eeae Se 220 80 & 


{4 ) Industrial Education 


To 


To 


To 


develop skills for entry into an occupation. ......eesseees cc ececcs 
provide practical application of academic knowledge. .--+-+e+---e-- 


develop basic skills to enter either a job or a post high school 


institution. ecoecvneeveceeeeevneeaeeeceeeerwmeseeneecteeeeeneeveeeen eee eee ee Ha eecoee soeaoaae 


To 


develop appropriate attitudes and work habits. ......-coeseoecceces 
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4 ) Language Arts 
To develop the ability to communicate clearly in speech and writing. .« 
To develop an understanding of language arts for personal enjoyment. . 


To develop a personal value system through examination of the value 
BSUCMS OLMOENDETS 2 6 oc ccc cc cme c00e case nese se ccsiccecccccoascconcercece 


To appreciate style changes in the English language. ............----- 
4 ) Second Language 
To develop an understanding of the people whose language is studied. . 


To appreciate the contributions of the people whose language is 


SS ERiMeClMlclclelalels sic slele cele © o.6.<)a\o «ele cleisicieinicc ce © ale 6eslelcccece cece cs cieciccec cc 
To understand the structure and use of the language. .eccccoes en eee cee 
To understand the language when spoken at a normal speed. ...........- 
To speak the language well enough to communicate. -..ceeeeccer ee eee eee 
To read in the target language with understanding. ....-.c2.2--2--22--- 
To write anything one can say in the language. ....----eec eee eee eee eee 


4 ) Mathematics 


To appreciate the contribution of mathematics tO progress. ........-.. 
To develop precision in thought and expreSSiON. ..cccrecceecrccccreece 
To understand and apply mathematical operations and concepts. ........ 
To develop the ability to understand and solve a problem. ..........-. 


1)” fusic 


To develop an appreciation of music past and present. ............---- 
To understand ways of communicating through music, ......--2-.-es-c2eee 
To recognize ones! own musical abilities. ....c.scccsccccce cee cccscoes 
To become aware of the basic importance of music in life. .......0--.. 
To increase ones! self-confidence. cccccccccccccccccecceccasecscsscccs 
{ L Physical Education 

To develop a Strong DOdY. cecccsecesccercccccsecccccccccccscccceccccce 
To develop recreational Skil1S. ...ccececccecccccccccccccccccedeceeccs 
To develop an interest in physical activities for leisure time. ...... 


To develop the ability to get along well with others. .....csesccecees 
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tee cLtence 
To understand the role of science in societies' development. .....-..-. 
To understand moral and ethical problems in science. .....cccceecceeee 


To understand social problems that are either caused, or solved 
through science. e@eeenecen eee eeeeeeeveeeaeeeneeeeneeneneeeaoeeenoce eeee eeeseaneaeene 


Pamirraenstandsandsuse SCIleENtific methodS ma .ccicccececcacccecccacscocece 
Prmrcecarnerundamental SCLENtific 1deasSc scccccccceccoeecuccecccoscesaes 
To develop scientific attitudes, interests, and values. ..c.csecccceces 


To connect scientific knowledge with vocational skills and 


opportunities. e@eeeeeoeeaceecn eee eeesneen eeeeneeneeceee eevee eneneea ee eee e eee eeneae2¢ 
4 } Social Sciences 
Lomunderstand basic» social science concepts. «ce -cccccccccenc sc cccecc esc 


To understand modes of inquiry and skills unique to the social 


SCIENCES. cece cece ccc cece ccc cece ccc ccc ccc ccc ccc ccc ccc crc ccc ccc ccccccce 
To understand how knowledge is produced in a particular discipline. .. 
i 1 Social Studies 

To develop an ability to choose from all known alternatives. .....-.-. 
Tosacerend ones*) choice Of alternatives. cjccccciaisclesicciclccices ces ciccce «6 
TomUakenmaction onetne chosen alternativescus.< cclccccicclccice cae ccs cia ss 
RamgevelOpeanavareness Ota Values circic acc ciclcle sicicle clelcicicicicicicciclecliciccicces se 
Toeehoo Sema mOnGeanVal let YeOLeval UCS cilccisuiasiaiciciciciciccicicicicciciciccciclec cles ese 


To develop a consistent, defensible system of values based on those 


choices. eeeosveveeneeeseee2esoe eee eemeeeeseeeeneeeeeeeeneaeneeneceeee2 ea Cee ene eae eee 
To recognize information relevant to social problems. ...........-.... 
To use information to propose a solution to social problems. ......... 


To respond to the feelings and ideas of others without compromising 
individual value system. eecre2eecenee ee eeeee2eenecesewzneeseeeeneeee2e 208 28920882 2080 
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‘APPENDIX C: Detailed Description of Variables from CEP Data Bank 
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DETAILED DESCIPTION OF VARIABLES 
SELECTED FROM CEP DATA BANK 


"General'' Outcome Variables - External 


(Source: Followup questionnaire in 1977) 

These variables are general in the sense of applying to the high 
school curriculum as a whole, rather than specific subject areas. They 
are external in the sense of being collected after the student has 
left the institution, via a followup questionaire five years after 
leaving school. : 

a. Student report of educational and career achievements: (n=725) 

V1- Type of further education programs involved in. 
V2 -— Number of years of further education. 

V3 — Employment status. 

V4 - Type of employment. 

V5 = Status of industry employed in. 

V6 — Wage bracket. 

Vies Level of satusfactaonywith, canpeer. 

b. Student-client evaluation of the adequacy of the school 

program generally, in reference to the achievement of 


PLovinGial CUuUcrACulL ume goals: (n=725/2). 


V8 - Evaluation of: "Develop the ability to use information 
and counselling services related to career decisions." 


V9 - Evaluation of: "Develop skills in managing natural, 
financial and human resources." 


V10 - Evaluation of: "Develop an understanding of functions, 
responsibilities and achievements of various societal 


institutions <><" 


Vil - Evaluation of: "Develop special interests and 
abilities." 


V12 - Evaluation of: "Develop a positive attitude toward 
learning." 


V13 - Evaluation of: 'Develop an understanding of good 
physical and mental health practices." 


Vi4 - Evaluation of: "Develop ability to apply scientific 
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methods in the pursuit of and analysis of knowledge." 


V15 - Evaluation of: "Develop an attitude of respect for 
public and private property." 


V16 - Evaluation of: "Establish sound personal health habits." 


V17 - Evaluation of: "Develop a feeling of cultural identity 
and heritage at national and international levels." 


V18 - Evaluation of: "Develop a positive attitude toward 
participation in a range of leisure time activities — 
physical, intellectual and creative." 


V19 - Evaluation of: "Cultivate appreciation for beauty in 
various forms.” 


V20 - Evaluation of: “Develop an awareness of and the ability 
to adjust to a changing social and physical environment." 


V2Z1 — Evaluation of: “Develop ability to adjust to the 
changing demands of Canadian society." 


V22 - Evaluation of: "Develop an awareness of civic rights and 
responsibilities." 


V23 - Evaluation of: “Develop intellectual curiosity and 
eagerness for lifelong learning." 


V24 - Evaluation of: "Establish a good physical fitness 
program." 


v25 - Evaluation of: "Develop understanding of the past, 
identify with the present and the ability to meet the future." 


V26 - Evaluation of: "Develop an understanding of the Canadian 
and other forms of government." 


V27 - Evaluation of: "Develop interests which will lead to a 
wise and satisfying use of leisure time.” 


V28 - Evaluation of: "Develop an understanding of the 
obligation and responsibilities of Canadian and world 
citizenship." 


v29 - Evaluation of: "Develop appreciation and respect for the 
worth and dignity of individuals." 


V30 - Evaluation of: "Develop an understanding of economic 
principles and responsibilities." 


V31 - Evaluation of: "Develop creative self-expression through 
various media including the fine and practical arts." 


PAY 


V32 - Evaluation of: "Develop understanding and skills in the 
use of numbers, natural sciences, mathematics and social 
sciences." 


Va 5— Evalnation ofiv Develor skill in oral and written 
languages." 


V34 - Evaluation of: "Develop a fund of information and 
concepts." 


V35 - Evaluation of: "Develop skills of thinking and 
proceeding logically." 


V36 - Evaluation of: "Develop ability in communicating’ ideas 
and feelings effectively." 


V37 - Evaluation of: "Develop special talents in the arts." 


V38 - Evaluation of: "Learn to take into account the values of 
others when making personal choices." 


V39 - Evaluation of: "Develop skill in understanding the 
communication of others." 


V40 - Evaluation of: "Develop ability to organize information 
into meaningful categories.” 


V41 - Evaluation of: "Develop skills basic to the world of 
work." 


"General" Outcome Variables - Internal 
(Source: Analysis of Dept. of Education student records) 
These variables are internal in the sense of being collected 
while the student is still within the institution. 
V42 - Average of high school marks. 
V43 - Level of high school achievement. 
vV44 - High school credit total. 
vV45 - Achievement of "100-credit'' requirement for high school diploma. 


vV46 - Achievement of "English'' requirement for high school diploma. 


V47 - Achievement of "Social Studies" requirement for high school 
diploma. 


v48 - Achievement of "Mathematics" requirement for high school 
diploma. 
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V49 - Achievement of "Physical Education" requirement for high school 
diploma. 


V50 - Achievement of "Science'' requirement for high school diploma. 


V51 - Achievement of “senior course'' requirement for high school 
diploma. 


"Student Based" Transactional Variables 

(Source: analysis of Dept. of Education student records) 

These variables are descriptive of the "process" or educational 
experience of the student in terms of courses taken. Variables 
collected were categorizations based on clustering of students 
according to their similarity of school program. 

V52 - Type of high school program followed (4 categorizatons). 
V53 - Type of high school program followed (15 categorizations). 
V54 - Type of high school program followed (9 categorizations). 
"Institution Based" Transactional Variables 

(Source: Analysis of school “A" cards) 

These variables are descriptive of school characteristics 
attended by the students in the sample. These variables relate to the 
on-going operation of schools, but for the purposes of this study have 
been delimited to variables which have potential for change through 
Scnool Board level policy or resource allocation decisions. Such 
variables are used for the purposes of dissaggregating outcome 
variables and where relationships can be established, may be used as a 
“third level" quality indicator. 
vV55 - School size: credits offered. 

V56 = School Sizes Studene enrolments. 
V57 - Number of.teachers in school. 


vV58 - Pupil-teacher ratio of school. 
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V59 - Number of counsellors in school. 
V60 - Type of school, based on programs offered. 
V61 - Type of term offered. 
"Student Based" Antecedent Variables 

(Source: Dept. of Education student records) 

These variables are considerd as "givens'' to the school and not 
Subject to change DYN LPOLUCVROLMeU marten Mmenanges:. 
V62 ="Student year of bireh: 
V63 —- Sex of student. 
V64 - Grade nine ability as measured by the Verbal SCAT score. 
V65 - Grade nine ability as measured by the Quantitative SCAT score. 
V6G,—a  Gradesnine Total —sCAL score. 
V67 - Grade nine departmental Reading score. 
V68 - Grade nine departmental Literature score. 
vV69 - Grade nine departmental Language Arts score. 
V70 - Grade nine departmental Social Studies score. 
V71 - Grade nine departmental Mathematics score. 
V72 - Grade nine departmental Science score. 
V73 - Average of grade nine departmentals. 
Vijte—ne Student sselt abiiiscyerating. 
V75 - SES as measured by student report of father's educational level. 
V76 - SES as measured by student report of mother's educational level. 
V77 -— Student report of having made a career decision or not. 
V78 -— Student report of future educational and career plans. 
"Institution Based" Antecedent Variables 


(Source: analysis of school "A" cards) 
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These variables identify the school, school district and 
categorizations of provincial regions. 
V79 -— Geographic region of school. 
VSO" =] Unban, small urban. of wsunaleschool. 
V81 -— Code of school attended. 
VE2ZNeCOdel of School di StEEICce, (division Or COUNTY attended 
"Subject Area'' Outcome Variables - External 

(Source:followup questionnaire in 1977) 

These variables focus on subjective evaluations of the 
achievement of objectives by each of the subject areas within the 


DEOvVINnecratenigh School cuir Guum. 


V83 - Evaluation of Art objective: "To develop an appreciation of 
abEUSEreEedesign. 


V84 - Evaluation of Art objective: "To develop an understanding of 
past and present art." 


v85 - Evaluation of Art objective: "To apply artistic design 
principles in art and everyday living." 


V86 - Evaluation of Art objective: "To develop individual artistic 
co aes 


V87 - Evaluation of Business Education objective: ''To develop an 
understanding of the operation of a business." 


V88 - Evaluation of Business Education objective: "To develop useful 
business knowledge and skills." 


v89 - Evaluation of Business Education objective: "To develop business 
skills of individul interest." 


v90 - Evaluation of Business Education objective: "To develop 
individuality in thought and action." 


v91 - Evaluation of Business Education objective: "To develop problem 
solving abilities." 


v92 - Evaluation of Business Education objective: "To apply business 
skills to other areas." 


V93 - Evaluation of Business Education objective: "To give practical 
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experience in the application of business skills." 
v94 - Evaluation of Drama objective: "To develop concentration." 
V95 - Evaluation of Drama objective: "To develop sensory distinction." 


V96 - Evaluation of Drama objective: "To obtain freedom and control in 
physical movement." 


V97 - Evaluation of Drama objective: "To encourage further interest in 
drama." 


V98 - Evaluation of Drama objective: "To communicate, through drama, 
an awareness of the current world issues." 


v99 - Evaluation of Industrial Education objective: "To develop skills 
FOBBeCNERVMI nto an occupatronces) 


V100 - Evaluation of Industrial Education objective: "To provide 
practical application of academic knowledge." 


Vi01 - Evaluation of Industrial Education objective: ''To develop basic 
Skills to enter either a job or a post high school institution." 


Vi02 - Evaluation of Industrial Education objective: "To develop 
appropriate attitudes and work habits." 


V103 - Evaluation of Language Arts objective: "To develop the ability 
to communicate clearly in speech and writing." 


V104 - Evaluation of Language Arts objective: "To develop an 
understanding of language arts for personal enjoyment." 


Vi05 - Evaluation of Language Arts objective: "To develop a personal 
value system through examination of the value systems of others." 


Vi06 - Evaluation of Language Arts objective: "To appreciate style 
changes in the English language." 


V107 - Evaluation of Second Language objective: "To develop an 
understanding of the people whose language is studied." 


vVi08 - Evaluation of Second Language objective: "To appreciate the 
contributions of the people whose language is studied." 


vio9 - Evaluation of Second Language objective: "To understand the 
structure and use of the language." 


Vi10 - Evaluation of Second Language objective: "To understand the 
language when spoken at a normal speed." 


Vi11 - Evaluation of Second Language objective: "To speak the language 
well enough to communicate." 
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V112 - Evaluation of Second Language objective: 'To read in the target 
language with understanding." 


V113 - Evaluation of Second Language objective: "To write anything one 
can say in the language.” 


Vi14 - Evaluation of Mathematics objective: "To appreciate the 
contribution of mathematics to progress." 


V115 - Evaluation of Mathematics objective: "To develop precision in 
thought and expression." 


V116 - Evaluation of Mathematics objective: "To understand and apply 
mathematical operations and concepts." 


V117 - Evaluation of Mathematics objective: "To develop the ability to 
understand and solve a problem." 


Vii8 - Evaluation of Music objective: "To develop an appreciation of 
music past and present." 


V119 - Evaluation of Music objective: "To understand ways of 
communicating through music." 


V1i20 - Evaluation of Music objective: "To recognize ones' own musical 
abilities." 


Vi21 - Evaluation of Music objective: "To become aware of the basic 
importance of music in life." 


Vi22 - Evaluation of Music objective: "To increase ones' 
self-confidence." 


V123 - Evaluation of Physical Education objective: "To develop a 
strong body." 


V1i24 - Evaluation of Physical Education objective: "To develop 
recreational skills." 


V125 - Evaluation of Physical Education objective: "To develop an 
interest in physical activities for leisure time." 


V126 - Evaluation of Physical Education objective: "To develop the 
ability to get along well with others." 


Vi27 - Evaluation of Science objective: "To understand the role of 
science in societies' development." 


V128 - Evaluation of Science objective: "To understand moral and 
ethical problems in science." 


V129 — Evaluation of Science objective: "To understand social problems 
that are either caused, or solved through science." 
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V130 - Evaluation of Science objective: 
scientific methods." 


V131 - Evaluation of Science objective: 
scientific ideas." 


Vi32 - Evaluation of Science objective: 
attitudes, interests, and values." 


V133 —- Evaluation of Science objective: 


"To understand and use 


"To learn fundamental 


"To develop scientific 


"To connect scientific 


knowledge with covational skills and opportunities." 


V134 - Evaluation of Social Sciences objective: "To understand basic 


social science concepts." 


V135 - Evaluation of Social Sciences objective: ''To understand modes 
of inquiry and skills unique to the social sciences." 


V136 - Evaluation of Social Sciences objective: "To understand how 
knowledge is produced in a particular discipline." 


Vi37 - Evaluation of Social Studies objective: ''To develop an ability 


to choose from all known alternatives." 


V138 - Evaluation of Social Studies objective: "To defend ones' choice 


of alternatives." 


V139 - Evaluation of Social Studies objective: "To take action on the 


chosen alternatives." 


V140 - Evaluation of Social Studies objective: "To develop an 


awareness of values." 


Vi4i - Evaluation of Social Studies objective: "To choose among a 


variety of values." 


V142 - Evaluation of Social Studies objective: "To develop a 
consistent, defensible system of values based on those choices." 


V143 - Evaluation of Social Studies objective: "To recognize 


information relevant to social problems. 


V144 - Evaluation of Social Studies objective: "To use information to 


propose a solution to social problems." 


V145 - Evaluation of Social Studies objective: "To respond to the 
feelings and ideas of others without compromising individual value 


systems." 


V146 - Art (N=305) mean rating. 


v147 - Business Education (N=375)mean rating. 
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V155 


V156 


V157 


Drama (N=210) mean rating. 
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Industrial Education (N=288) mean rating. 


Language Arts (N=590) mean rating. 


Second Language (N=468) mean rating. 


Mathematics (N=690) mean rating. 


Music (N=208) mean rating. 


Physical Education (N=660) mean rating. 


Science (N=655) mean rating. 


Social Sciences (N=343) mean rating. 


Social Studies (N=660) mean rating. 


(Source: Analysis of Dept. of Education student records) 


These variables are descriptive of individual student 


achievements within each of the subject areas. 
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Business Education. 
Home Economics. 
Industrial Arts. 
Vocational Education. 
BEDE 

Occupations. 
Perspectives for Living. 
Driver Education. 
Religion. 

Special Projects. 
matriculation English. 
general English. 

Social Studies. 

Social Sciences. 
mMatriculaton Mathematics. 
general Mathematics. 
matriculation Sciences. 
general Sciences. 
Second languages. 

Atala. 

Drama. 
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Home Economics. 
LNdUSthial Arts. 


Vocational Education. 


Average of 
Average of 
Average of 
Average of 
Average of 


Average of 


Grade 


Grade 


Grade 


Grade 


marks in 
marks in 
Marks in 
marks in 
marks in 
marks in 
distribution 
dagistribution 
distribution 
aistribution 
distribution 
GiSteri bucson 
distribution 
distribution 
distribution 
distribution 
distribution 
distribution 
distribution 
distribution 
distribution 
distribution 
distribution 
distribution 
distribution 
distribution 


distribution 


HPD. 


Occupations. 


Perspectives for Living. 


Driver Education. 


Religion. 


Special Pre jeces. 


(A086, C, UF) 
CEE. Cruee) 
CAS B.C, Dk) 
(A,B,C,D,F) 
(Aen. ry) 
CAB CeD. F) 
(AS RC. D.F) 
(4B, C.DeF) 
(AS. C.D.) 
CAP Be Ce) ED 
(AUB,C.D,F) 
(A BECO B) 
(A,B,C, 0,F) 
(A,B,C, D;F) 
(A,B;C,D,F) 
(ABC. Ue) 
(AYE C.D) 
(A,B,C, De FY 
(As. CoD.) 
(A,B,C,D,F) 


(AUB, C.D eF) 


18} 


in 


in 


in 


alsa 


in 


in 


in 


akin 


in 


in 


in 


aig) 


in 


aid) 


skid 


in 


in 


Siig! 


in 


in 


matriculation English. 


general English. 


Social Studies. 


Social Sciences. 


matriculation Mathematics. 


general Mathematics. 
matriculation Sciences. 
general Sciences. 
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V225 - Grade distribution (A,B,C,D,F) in Religion. 


V226 - Grade distribution (A,B,C,D,F) in Special Projects. 
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